actual graphic-based screen

You have to be much more specific than that, else I just say that there exist tools and modules for inspecting and automating GUI applications, and it depends.