identifying web UI elements ... alternate approach

[Walter.Koller]

When exchanging experiences with the team of another (non-RPA) automation tool we got an idea that might improve how BP identifies (web) UI elements.

Currently BP seems to use the absolute path of an element for identification and has to parse the whole UI / HTML page to find this element again. The identification consists of the 'path' to the element and the attribute of the element. 

The disadvantage is the increased time to parse the whole document (therefore using matchIndex can bring big improvements) and failures in case the actual path to the element differs to what BP expects.

An alternate approach is to use the hierarchical organization of element. The difference would be:

absolut path might look like: /HTML[1]/BODY[1]/MWD39-MODAL[1]/DIALOG[1]/DIV[1]/NG-COMPONENT[1]/DIV[2]/DIV[1]/FORM[1]/DIV[1]/SELECT[1]

If any order changes (eg DIALOG changed from 1 to 2) the element will not be found.

The alternative would be to identify the element in steps with each step would be able to have its own identification. eg go to MWD39-MODAL, find DIALOG as defined by..., find DIV with NG-COMPONENT, ....

This way it would not matter if DIALOG (or any other part of the path) is [1] or something else since it is identified by more attributes than a simple number.

[John__Carter]

I think you can achieve this flexibility using an Xpath expression for the Path attribute (and de-selecting all other attributes).

[Walter.Koller]

Maybe I failed in explaining what the idea is all about.

Currently the usage of XPath is to pinpoint one element with its entire 'constant' path. 'constant' in quotes since some dynamic attributes can be added. However dynamic attributes add more complexity to the process during runtime.
I will try to explain the idea better by example (I am not a HTML expert so the example might not technicall perfect but should only present the concept).
<table>
  <tr>
    <th>Month</th>
    <th>Savings</th>
  </tr>
  <tr>
    <td>January</td>
    <td>$100</td>
  </tr>
  <tr>
    <td>February</td>
    <td>$80</td>
  </tr>
</table>

XPath might look similar to: '<table>[1]/<tr>[2]/<td>[1]' to get the cell with 'January'.
If the January switched place with February, XPath will fail. 
If somewhere at the path eg <div> is added, it will fail.
If the table is not the first on the page anymore, it will fail.
...

The proposal is to identify and process each 'container' on its own to get to the desired element. eg
'find the table with header 'Month''
'find the line with 'January' in this table'
...
So it won't matter if the table is the first on the page or one of x other tables. Also I don't have to care the order of the table content. asf

Probably a better example would be a change in <DIV> tags that (as far as I know) has purely organizational purpose but won't change the actual structure of the page. However, a change in <DIV> might break XPath.

Or to use the terminology of the application I became aware of this feature: Identify controls by anchor

[John__Carter]

Hi Walter - this Xpath expression is an interpretation of 'find the table with Month in it, find the cell in that table containing January, then find the next cell in the same row

//th[text()='Month']/ancestor::table//td[text()='January']/following-sibling::td

Xpath is highly flexible and lets you travel up and across the HTML model. Critically, it enables you to find elements in relation to other elements, so in the example above, it doesn't matter where the table is in the page, as long as it has a Month header and a January cell, the path will work.

There is a ton of Xpath material out there but I've found these 2 tools to be useful.
https://devhints.io/xpath

http://xpather.com/

[Walter.Koller]

Thanks a lot of the quick intro in XPath features 🙂

I will forward this to our developers 

[Walter.Koller]

...but it would be nice if Blue Prism would support the users in this approach.

You know, we are talking about next gen automation, increased usability, creating more robust automation, how to integrate AI, ...