The value from 'Get Attribute' Web Y is different when used in Action 'Global Mouse' Click Y

[JeremyRTDean]

I'm using Browser automation, using Edge and BP v7.1.1.

If I use 'Get Attribute' and the Web Y attribute to get a Y-axis number value.

Then use that Y-axis number value in the action: 'Global Mouse' Click Y to select the row.

It clicks on a different row.

I can work around it in this case by adding 90 to the Y value and it will click on the correct row (so a calculation: [Web Y]+90)

But the question is, why does the Y-axis value read not give the same row when I use it in an action?

[devneetmohanty07 · Accepted solution]

Hi @JeremyRTDean ,

What you are facing is because of the fact that Global Mouse Click is a Win32 or an UIA based method but the attribute that you are getting is using Browser Mode. Every spying mode in Blue Prism works with respect to it's in-built technology framework and they are unique from each other.

When you are spying using Browser mode, the attributes are relative to an HTML document of the website but when you spy using UIA or Win32 they are relative to the Windows container.

For example, if you see below, I am spying the same element using two spying modes but I get different co-ordinates:

Now the question is then which one should you sue with Global Click action. I would say mostly any co-ordinate would work, only difference being UIA or Win32 spied elements will give you the exact co-ordinate for the top, left, right and bottom section of that element while with Browser spied elements you will have to adjust the value as you have done above. But in my experience, Web element co-ordinates are not always reliable especially if they are in an IFRAME element. So, if you have to rely on a co-ordinate based approach, go for UIA or Win32 or AA mode as they work with relativity to the screen and Global Mouse Click would work pretty well with that. With this being said, UIA can impact performance by a lot in some cases, but accuracy would be good while Browser accuracy is better but is not reliable for a co-ordinate based automation. So, you should test and decide which one works the best for you at the end of the day.

Again, remember the co-ordinates that we get would be for the edge of the element so technically if you want to click on the center you will have to add some offset no matter which mode you use as shown below:

[JeremyRTDean]

Excellent answer.  The reason I had to do this was because no BP action worked to select a checkbox in this application.  The most reliable and simple way to select the row was to use a web element but the Global Mouse Click needs a UIA element so the only way I could think to do it was use a mix of both and incorporate the difference in coordinates with that +90 offset.  Seemed to work OK but a little complicated.

[John__Carter]

I think the easiest (or least awkward) way is

• Spy the browser inner window/web content area as Win32
• Scroll the web element into view if necessary
• Read the web element's coordinates with Get Bounds
• Activate the browser outer window and use the coordinates to click the inner window element

You can abstract the click to a new page (or even a separate utility object) for reuse, like this.