Can Blue Prism provide the functionality of Click Text as provided by UI Path function

[ManishRaw]

In UI path if the element is not identified then the second level of identification is text identification and they have inbuilt functions like "Click Text",  am not sure if BP provides a functionality like that or not. If its there then please tell me how we can use that and if not then I think it could be a great enhancement.

[david.l.morris]

Hi Manish, I don't think there's exactly the same functionality as 'Click Text' provides. Do you happen to know how that UiPath function finds the text and clicks on it? Is it using screen scraping to find the coordinates of the text? What you're asking for can be extremely easy in Blue Prism, depending on the situation. I'm going to assume a webpage just for my example, but this could apply to other application types. For example, if you know that there will be an element on the page that exactly equals the text you're looking for, then you can simply look for any element that equals that text and click on that element. Let's say you want to click on a button in a webpage that exactly says 'Subscribe'. In application modeller, you could set up your back-up identification element that only identifies using (perhaps) the attributes 'Value' and 'Match Index'. You set Value to 'Subscribe' and Match Index to '1'. Both of these attributes' match types would be set to Equals because it's likely the word subscribe could show up elsewhere on the page as part of a text element. Then you use a navigate stage in your object to do a Global Mouse Click Centre on that element. The reason I could see this being useful is if you originally use the DOM/HTML Path as part of the identification but then at some point the HTML of the page changes enough that the button is no longer located in the same place. This is also assuming that the button does not have a unique identifier. Now, the above is assuming that you know there will be an element that has its 'Value' attribute equal to something exactly. If you're trying to click into the middle of a paragraph of text because perhaps you know a set of text will be a link, that's going to be more difficult. It is definitely still possible but it's certainly not as easy as the UiPath activity/funciton 'Click Text' or even the above-described method of clicking on an element anywhere on the page where the value attribute equals 'Subscribe'. Let me know what you think. I've never had a use for this, but I can see the need if the app is highly dynamic. Respectfully, Dave

[ManishRaw]

Hi Dave,   The approach provided looks good in case we are able to identify elements. But the issue I have is that I have a window in desktop based application and the same has text spread across it. The application exposes the text data to Blue Prism and UI Path but there is no way that I can directly click onto the text provided in that window using Blue Prism but UI path has that capability with that Click Text property. So I was just thinking that if Blue Prism is able to identify the text will there be a possible solution to click the text as well. P.S.: Not exactly sure what method does UI Path uses internally for that Click Text property but yes they first screen scrape it and then find what text is available and then click over it. So they have a 4 level approach:   1. Identify element 2. Identify Text 3. Identify Image 4. Identify Imange with OCR.   So in my scenario we are able to do the work using level 2 approach as elements are not visible but the text inside the window is visible.   Let me know in case there is some way if I can select the whole window as an element in any of the mode, Window, UI or AA and then click over the text exposed by the application.   Regards, Manish

[david.l.morris]

So, am I understanding right? You basically have a paragraph of text similar to what I'm typing now and you need Blue Prism to click (for example) *this text* but because that text isn't an element by itself, Blue Prism just sees the entire paragraph as text or maybe the entire body of text as one big element? And I assume if you tried to click on it using a navigate stage with ""global mouse click centre"" that it would just click into the middle of the paragraph instead of on the text that you need to click on. Is that right?

[ManishRaw]

Yeah David you are correct with the problem statement. 

[david.l.morris]

Are you able to attach a screenshot of the text? Unless there's something that I'm forgetting, I don't think the same Click Text feature is available in BP. That'd be a good feature enhancement to request. There's probably a way to accomplish it using Surface Automation techniques. For example, if that text is a different color from the rest of the text, you could look for a small image that is entirely made up of that color and then you can click where that is found. That doesn't sound super robust to me, but I'd be interested in seeing a sample of the paragraph containing the text you need to click on. Dave

[AmiBarrett]

The closest we've been able to do is with image matching on a custom-built BO. Whatever method BP natively uses for its Region Mode to accomplish this are incredibly slow by comparison. Anyway. If you can compare by sub-image (Take a screenshot and find a smaller image containing the text you're looking for within it), you can get the coordinates of that. From there, you can use code to click said coordinates. We're using some of this today in Oracle to bypass the need for most of the JAB integration, due to how slow it is. While this method isn't the speediest thing ever, it's certainly faster that what BP provides out of the box.