Simulating mouse clicks in Windows

tall_man has asked for the wisdom of the Perl Monks concerning the following question:

I am trying to communicate with a perl/Tk status bar from another application in Windows NT. I thought I should be able to bind an action to a mouse click and then send simulated mouse clicks to it from my application. Then the status bar would be up-to-date with the current state of the application without interfering with it. If possible, I want to simulate mouse clicks to the target window without actually grabbing and moving the mouse.

I have tried two ways of doing it and both are not working: Win32::GuiTest and Win32::CtrlGUI.

Here is the first way, using Win32::GuiTest.

use strict;
use Win32::GuiTest qw(FindWindowLike GetChildWindows GetWindowText
                      GetWindowID SetActiveWindow SetForegroundWindow
                      PostMessage);

sub WM_LBUTTONDOWN      { 0x0201; }
sub WM_LBUTTONUP        { 0x0202; }

my @winlst;
my $tries = 0;
while ($tries < 20) {
   print "trying to find the window\n";
   @winlst = FindWindowLike(0,'StatusBar','TkTopLevel');
   last if scalar(@winlst) > 0;
   $tries++;
   sleep 1;
}
if (@winlst == 1) {
   my $status = $winlst[0];

   my $i;
   my $x = 30;
   my $y = 14;
   my $xy = ($y << 16) + $x;
   print "trying to send a click to the window\n";
   SetForegroundWindow($status);
   SetActiveWindow($status);
   PostMessage($status, WM_LBUTTONDOWN, 0x0001, $xy);
   PostMessage($status, WM_LBUTTONUP, 0, $xy);
}
[download]

Here is the second way, using Win32::CtrlGUI.

use strict;
use Win32::CtrlGUI;

my $win = Win32::CtrlGUI::wait_for_window("ParpadStatus");
my $x = 30;
my $y = 14;
my $xy = ($y << 16) + $x;
print "trying to send a click to the window ",
   sprintf("0x%lx",$win->handle),"\n";
$win->post_message('NN', "WM_LBUTTONDOWN", 0x0001, $xy);
$win->post_message('NN', "WM_LBUTTONUP", 0, $xy);
[download]

In both cases Spy++ shows that the window is getting the button events, but it isn't responding. There are also some unusual messages in the log in both cases about a message 0x118:

003000BE R WM_ACTIVATE
003000BE P WM_LBUTTONDOWN fwKeys:MK_LBUTTON xPos:30 yPos:14
003000BE P WM_LBUTTONUP fwKeys:MK_LBUTTON xPos:30 yPos:14
003000BE P message:0x118 [Unknown] wParam:0000FFF8 lParam:A00885B9
003000BE S WM_NCACTIVATE fActive:False
003000BE R WM_NCACTIVATE fDeactivateOK:True
[download]

Does anyone have ideas about how I can get this to work, or if I should abandon it for better ways? I would rather not use sockets, because later I want to communicate asynchronously with other more complicated perl/Tk applications.

Comment on Simulating mouse clicks in Windows Select or Download Code

Replies are listed 'Best First'.
Re: Simulating mouse clicks in Windows by Mr. Muskrat (Canon) on May 29, 2003 at 19:10 UTC
Using Win32::GuiTest, you have some cool functions to work with. You could use MouseMoveAbsPix and SendMouse in conjunction, use PushButton, or bind a keystroke to the button and use SendKeys. :-)	[reply]
Re^2: Simulating mouse clicks in Windows by tall_man (Parson) on May 29, 2003 at 19:30 UTC
Thanks for your suggestions, Mr. Muskrat. I considered solutions where I forced the mouse to move with MouseMoveAbsPix, but I would rather simulate the click events without moving the mouse if I can. The user would be distracted and annoyed if I take mouse control away. My status window is just a simple one with a Tk::ProgressBar and a text string in it, so PushButton won't work. SendKeys may be my best option. Keystroke events are a bit of a pain because sometimes another window takes the focus and gets them, but they may be the best I can do.	[reply]
Re: Simulating mouse clicks in Windows by BrowserUk (Patriarch) on May 29, 2003 at 20:22 UTC
Have you tried capturing the sequence of messages sent and recieved when you click the button with the mouse and then emulating that sequence? I'd start capturing when the app doesn't have focus, and the pointer is off the apps window. Then bring the mouse over the window to the button and click. If seen dialog procedures for controls that only start taking notice of msgs once they receive a mouseover msg, and the stop again once they receive a mouseout. If you started by sending the full sequence captured and it works, then you can start omitting stuff until it stops working again. I've never tried this from perl, but it's worth a shot maybe. Examine what is said, not who speaks. "Efficiency is intelligent laziness." -David Dunham "When I'm working on a problem, I never think about beauty. I think only how to solve the problem. But when I have finished, if the solution is not beautiful, I know it is wrong." -Richard Buckminster Fuller	[reply]
Re: Re: Simulating mouse clicks in Windows by tall_man (Parson) on May 30, 2003 at 00:14 UTC
Thanks, BrowserUK. I am trying your approach, but every little thing that is done in Windows produces an overwhelming flood of messages,so it's hard to sort the wheat from the chaff. Here is a sequence: <00010> 0003009C P WM_MOUSEMOVE fwKeys:0000 xPos:67 yPos:7 <00011> 0003009C P WM_MOUSEMOVE fwKeys:0000 xPos:47 yPos:5 <00012> 0003009A S WM_PARENTNOTIFY fwEvent:WM_LBUTTONDOWN xPos:87 yPos +:8 <00013> 0003009A R WM_PARENTNOTIFY <00014> 00030098 S WM_PARENTNOTIFY fwEvent:WM_LBUTTONDOWN xPos:92 yPos +:13 <00015> 00030098 R WM_PARENTNOTIFY <00016> 000200AE S WM_PARENTNOTIFY fwEvent:WM_LBUTTONDOWN xPos:92 yPos +:13 <00017> 000200AE R WM_PARENTNOTIFY <00018> 00030096 S WM_PARENTNOTIFY fwEvent:WM_LBUTTONDOWN xPos:92 yPos +:13 <00019> 00030096 R WM_PARENTNOTIFY <00020> 0003009C S WM_MOUSEACTIVATE hwndTopLevel:00030096 nHittest:HTC +LIENT uMsg:WM_LBUTTONDOWN <00021> 0003009A S .WM_MOUSEACTIVATE hwndTopLevel:00030096 nHittest:HT +CLIENT uMsg:WM_LBUTTONDOWN <00022> 00030098 S ..WM_MOUSEACTIVATE hwndTopLevel:00030096 nHittest:H +TCLIENT uMsg:WM_LBUTTONDOWN <00023> 000200AE S ...WM_MOUSEACTIVATE hwndTopLevel:00030096 nHittest: +HTCLIENT uMsg:WM_LBUTTONDOWN <00024> 00030096 S ....WM_MOUSEACTIVATE hwndTopLevel:00030096 nHittest +:HTCLIENT uMsg:WM_LBUTTONDOWN <00025> 00030096 R ....WM_MOUSEACTIVATE fuActivate:MA_NOACTIVATE <00026> 000200AE R ...WM_MOUSEACTIVATE fuActivate:MA_NOACTIVATE <00027> 00030098 R ..WM_MOUSEACTIVATE fuActivate:MA_NOACTIVATE <00028> 0003009A R .WM_MOUSEACTIVATE fuActivate:MA_NOACTIVATE <00029> 0003009C R WM_MOUSEACTIVATE fuActivate:MA_NOACTIVATE <00030> 0003009C P WM_LBUTTONDOWN fwKeys:MK_LBUTTON xPos:47 yPos:5 <00031> 00030096 S WM_WINDOWPOSCHANGING lpwp:0140FCC0 <00032> 00030096 R WM_WINDOWPOSCHANGING <00033> 00030096 S WM_WINDOWPOSCHANGED lpwp:0140FCC0 <00034> 000200AE S .WM_WINDOWPOSCHANGING lpwp:0140FBA0 <00035> 000200AE R .WM_WINDOWPOSCHANGING <00036> 00030096 R WM_WINDOWPOSCHANGED <00037> 00030096 S WM_ACTIVATEAPP fActive:True dwThreadID:00000000 <00038> 00030096 R WM_ACTIVATEAPP <00039> 00030096 S WM_NCACTIVATE fActive:True <00040> 00030096 R WM_NCACTIVATE <00041> 00030096 S WM_ACTIVATE fActive:WA_ACTIVE fMinimized:False hwnd +Previous:(null) <00042> 00030096 S .WM_SETFOCUS hwndLoseFocus:(null) <00043> 00030096 S ..WM_KILLFOCUS hwndGetFocus:000200AE <00044> 00030096 R ..WM_KILLFOCUS <00045> 000200AE S ..WM_SETFOCUS hwndLoseFocus:00030096 <00046> 000200AE R ..WM_SETFOCUS <00047> 00030096 R .WM_SETFOCUS <00048> 00030096 R WM_ACTIVATE <00049> 000200AE S WM_KILLFOCUS hwndGetFocus:000200AE <00050> 000200AE R WM_KILLFOCUS <00051> 000200AE S WM_SETFOCUS hwndLoseFocus:000200AE <00052> 000200AE R WM_SETFOCUS <00053> 0003009C P WM_MOUSEMOVE fwKeys:MK_LBUTTON xPos:47 yPos:5 <00054> 0003009C P WM_MOUSEMOVE fwKeys:MK_LBUTTON xPos:47 yPos:5 <00055> 0003009C P WM_MOUSEMOVE fwKeys:MK_LBUTTON xPos:47 yPos:5 <00056> 0003009C P WM_MOUSEMOVE fwKeys:MK_LBUTTON xPos:47 yPos:5 <00057> 0003009C P WM_MOUSEMOVE fwKeys:MK_LBUTTON xPos:47 yPos:5 <00058> 0003009C P WM_MOUSEMOVE fwKeys:MK_LBUTTON xPos:47 yPos:5 <00059> 0003009C P WM_LBUTTONUP fwKeys:0000 xPos:47 yPos:5 <00060> 0003009C S WM_CAPTURECHANGED hwndNewCapture:00000000 <00061> 0003009C R WM_CAPTURECHANGED <00062> 0003009C P WM_MOUSEMOVE fwKeys:0000 xPos:47 yPos:5 <00063> 0003009C S WM_WINDOWPOSCHANGING lpwp:0140F880 <00064> 0003009C R WM_WINDOWPOSCHANGING [download] What it looks like to me is that there is a cascade of WM_PARENTNOTIFY messages from the lowest child up the parent tree. Then there is a cascade of WM_MOUSEACTIVATE messages up the tree (each returning a MA_NOACTIVATE). I don't think the WM_WINDOWPOSCHANGING messages matter, but the WM_ACTIVATEAPP and WM_SETFOCUS messages at the top level probably do. Pretty messy, but I'll keep working on it.	[reply] [d/l]