Surface Automation

[MattDavis]

I'm having a little trouble with surface automation/region mode. From my understanding region mode captures a screenshot of the current region and then you define the regions you want to interact with based on that screenshot. But is there anyway to "lasso" the parent region that you want BP to look for rather than it trying to automatically detect the regions on the screen with the little red box?   The issue we run into is that the parent image isn't frequently found because the data on the screen changes and we only need to interact with a small part of this window. I read through the ATK and didn't see any reference to this. I know with UiPath it's super easy to "Click Image" as the engine doesn't have to find the parent screenshot, it just simply looks for the image on screen.   Any help or documentation you can point me towards would be appreciated 🙂 

[DaveMorris]

If you've read through the Surface Automation documentation, then I'm not going to go into detail here. But you may have missed the info about padding which can be used as the sort of alternate search location for images. I don't have the docs or BP pulled up right now so my wording may be off. But basically if you wanted to, you could set the padding for a region to encompass the entire screen. And then BP would first try to find the image in the middle of the defined element location and then would spiral outward looking for it. This is as opposed to as you suggested where it could instead be tied to a parent image. Also, I'm referring to BP 6 so if you're in BP 5, I'd suggest going on strike until your organization meets your demands of upgrading. 😃

[AshishEasow]

SA in BP works completely on images, one way if youwant to search the entire screen is how david mentioned, another is by searching for the parent first then try to find the child by some relative coordinates, since its an image, you will need some sort of relative image as parent.  In BP portal if you search for Surface Automation in documents, you should be able to find the ATK for it, it will show all the features available for your version and you can check if it works for you.

[JohnMonastra]

I will also add in my short time working with surface automation that it is very sensitive to any changes in the document itself or in the application that is presenting it. For example, in working with Adobe Acrobat Reader, you need to ensure that the if the window was maximized when originally performing the region mapping that it remains so in subsequent use of that modelling.  Also, things like magnification %, side bars being visible/invisible impact the results as well.