AA mode does not see some elements before a spy operation

[marco.pogliaghi]

Hi, I have this strange behaviour: in AA mode I've spied some elements and all worked well, but when I have run the process ex-novo, the elements were not reached any more (Unable to match any Active Accessibility elements to the supplied query terms).

If I go and make a spy operation (note: I just make a "Identify" operation in AA mode on a new element also without saving it), that the elements are reached again and all works well on thaty run.

I've made some test using only "Window Text" attribute and ciclying on index: I had the first 24 elements found and an "Unable to match any Active Accessibility elements to the supplied query terms" failure on the 25 and up. So I make an "Identify" operation in AA mode, and then I find also the other hundreds items.

Have you any hints?

[naveed_raza]

can you please share the application modeler screenshot of that element 

[marco.pogliaghi]

Hi Naveed, this is the screenshot when you can see the App modeler, the application on the background and the test result (I see I was wrong: the elements found where 24, the 24th is the first "not found" element)

[marco.pogliaghi]

The second part of the story: I just make a generic "Identify" operation, on a generic object, just to do an Identify op:

And after this I've relaunched the test, and this is what I obtain:

[naveed_raza]

looks like , its windows based application .

you can do one thing , once you launch the application , go to task manager , expand the process , there you can see the actual window text.

another thing is , if its windows based application then you can try with win mode to identify the window

[marco.pogliaghi]

This is a windows based application that embeds a web based application, formerly I reached the webbased application and could use the html elements, now they changed the way they embeds the web app and the html is no more reacheable, so I use UIA or AA, Win Mode does not reach any object in the application, it just sees the external frame:

What I don't understand is: why after an "Identify" operation the objects are visible? What changes? Is it something I can do in my code too?

[naveed_raza]

if you are getting windows from web app then you have to create a separate object to attach to that window, you should not use in web based application modeler .

we got this confirmation from our RPA COE team here, so when ever we get any windows based app inside web app then we are creating a new object (application already running) windows options to attach and then identifying the elements

[marco.pogliaghi]

Yes: this is a new object created for this application (well, for this specific windows in this application).

Also, the situation is different: not a windows app inside a web app, but a web app embedded in a windows app.

[marco.pogliaghi]

I've isolated the issue (even if I still don't have a resolution): my issue occurs when an application (in my case (IC.IntegratoreClient, a dot.Net custom application), run an embedded web application using msedgewebview2.exe program.

What happens is that the BPAgent "sees" only the first few items of the embedded app (I still Haven't udersttod which ones, I guess they are the object natively present in the webview window, but not the components of the web application). 

The "magic" thing that happes is that if I open an object, even another object, not the one that should interact with the fields of the application, just one that has "IC.IntegratoreClient" as application) and start an identify operation in AA Mode (even if I abort it and don't save anything), then all components of the webview application become visible for my object too and my applicastion can prosecute doing its job.

I'm using Blue Prism 7.2.1 version on a W11 virtual machine.

Have you any hint? Is this a known issue? Is there a workaround?

Thank you,

    marco

[Azarudeen_S]

Hi Marco,

I am happy to help with this. Instead of using AA mode, try switching to UIA mode to spy the elements. First, capture the element using UIA mode and select the unique attributes to identify it. Please use UIA mode for all your object modeler elements.  

In your case, start by testing with the Target Match Index. If the element still isn’t highlighted correctly, try using the UIA Text attribute to match the element more reliably.

Could you please let me know what exactly you are trying to achieve here? This will help me guide you more accurately.