Browser Automation - Global Mouse Click (Center)

[rsrivastava2]

Hi All,

With MV3 introduced in chromium and discontinuation of Internet Explorer, moving automation to Chrome or Edge was a solution. What I see is, you do not have Global Mouse Click (Center) option - even though we have Global Mouse Hover.

I am not a fan of UI Spying for time delays and I love how quick is XPATH, but currently I have to -

• Activate Window
• Focus Element
• Send Global Key Events -  {ENTER}

A Global Mouse Click (Center) is what is needed to Click and Element and also fire events.

If an alternative/work around is available, please comment here and share with larger group.

[BH.Hwang]

The best or not,
I used to Read and Navigate usefully as follows:

Read Stage

Navigate Stage

Hope this helps.

[rsrivastava2]

I really do like the work around BH.Hwang.

Though, this idea is to reduce the number of steps needed to mimic a global mouse click.

But, meanwhile, I am defiantly going to use your solution in the interim.

Thankyou!

[faheemsd]

Dear @rsrivastava2 

I understand that we don't have Global Mouse Click center in Browse mode and so many of Blue Prism professions are looking for this feature in the upcoming versions hopefully.

I agree with you that UI will give some Delay for clicking the element but it's good to go with UI for Global mouse click center instated of send key events because UI mode will mimic exact click center functionality.

[Mohamad_747]

Hello dear friends, I currently have the same problem. I have a button that is a div, with the ui mode it is impossible to spy on it because it only works the first time. I used the browser mode and precisely the x path which makes the processing much faster but on the other hand I can't click on it. With the X path I can retrieve the bounds with the read stage, but I noticed that when I spy this button with the Ui mode it doesn't give me the same coordinates (in addition to that when I try to review the element with the ui mode it doesn't work). I tried  @BH.Hwang solution, so i put the coordinates of x and y as he mentionned in the click window but I have the impression that it doesn't click on the requested element. Do you have any idea please ?

[BH.Hwang]

I can't find an answer because I don't know what the actual situation is. Assuming one thing, I think it could happen if the location you click is not the location visible in the Windows window. In this case, I think that if you use the value 1 instead of Width / 2, the element will be clicked even if only part of it is visible across the screen. One way to check whether a click is made is to test with "RIGHT" instead of "LEFT" and you can check whether it was clicked by showing the Context Menu.

[Mohamad_747]

Hi @BH.Hwang  So, I will give you more information :)

i have an element spied in browser because in UI Mode its not working. This is the element : 

T

i get the attribute web X and web Y with a read stage on my element to have an idea about the coordinates with this action :

i get for example  X = 9 and Y = 116

when i use the UI Mode (it work just for one time, i dont have the same X and Y) see here : 

and even when i use get bounds with the same element spied in browser i dont have the same X and Y coordinates so i cannot use the click window 

see here : 

we have only the 9 common between them but Y is changing so i cannot use your method here 

To resume what i said, i there any solution to click on element if we have an element spied with Browser and not with UIMode ? 

Thanks :)

[Asilarow]

Hi Mohamad,

I'm going to be a rebel here and say: try using Region mode instead.

It will do what you want, and will work really quickly :)

Just make sure to follow best practices for Region mode, and it will all be good.

[Mohamad_747]

Hi dear @Asilarow 

Its forbidden in my company unfortunetly :(

[rsrivastava2]

Hi All,

Thanks for all your suggestions.

The idea here though is to be able to use browser mode to do what it is supposed.

[BH.Hwang]

Hi @Mohamad_747 

UIA X, Y are coordinate values ​​starting from the origin of Screen. Blue Prism does not have the ability to click the absolute position of the screen, so if necessary, you will have to use Code Stage to create it. It seems possible to use the mouse_event() function in winapi, but can we find it on digitalexchange?

Since my method clicks based on the coordinates of the Window Control in the browser, using UIA X, Y does not conform to the concept of the Click Window action in Navigate Stage.

I also have a poor Blue Prism manual, so it's difficult to fully explain the concepts of Get Screen Bounds, Get Bounds, Get Relative Bounds and so on. In particular, the concept of Relative Bounds requires an accurate understanding of the concept of Parent Element, but I could not easily find good documentations.