Using Surface Automation to read Captcha from a website & Writing in word table using ms word vbo

[MahmodAbuSableh11]

Hello, I have 2 problems :
1. I want to read captcha using surface automation and it doesn't work , is there a way to achieve my goal by using surface automation or other way?
2. I want to write my collection to word table but i don't know how. (The table template exist)

[Daniel_Sanhueza]

Hello @MahmodAbuSableh11 

1.- As far as I know, you cant read those captchas with Surface Automation, the OCR that comes with blueprism isnt powerful enough, the nature of captchas is dynamic so it is very difficult to the OCR to read it. We just decide to not automate platforms that require captchas. Perhaps someone knows an API that can help to our RPA to read those captchas? Lets see others perspectives.

2.- If you already have a table, you can use the action "Set Table Cell value" the input "Table number" should be 1, if you have more than one table, you have to change depending on which come first. Create 2 number data item [X] and [Y] with his initial value in 1 and use it when is needed. Loop the collection you want to write and add 1 to X ([X]+1) when you write each cell, add 1 to Y ([Y]+1) when you wrote all the cells of the row.

If I remember correctly, each cell is like this;

{x,y}={1,1}	{x,y}={2,1}	{x,y}={3,1}	{x,y}={4,1}
{x,y}={1,2}	{x,y}={2,2}	{x,y}={3,2}	{x,y}={4,2}

hope it helps, regards

[devneetmohanty07]

Hi @MahmodAbuSableh11 ,

Since, @Daniel_Sanhueza has provided a response to your second query, I will shed some light on the first one.

To read Captcha, I would not suggest using any OCR solutions especially the Tesseract OCR that comes as part of the Blue Prism product suite, reason being that Captcha's are usually not your normal text which an OCR can directly read in 80% of cases since most of these text values would have a background image colour, or can have dynamic sizes, fonts and a lot of variations. You can however, try Microsoft Azure Computer Vision API services as I have seen it gave an accuracy of around 60-70% in one of my past proof of concepts. However, you may need to benchmark various OCR results together against this specific use case to assess which fits your needs the best both by functionality and licensing costs as most of the OCR solutions won't come by without a license cost.

Now, coming to the part if you can read a Captcha or not outside of OCR, the answer is both a yes and a no as it largely depends on the type of Captcha that is being used on the website end. For example, Google RECaptcha service is perhaps one of the most advanced and unbreakable CAPTCHA service starting from v3 onwards.

However, if your web site is using some other CAPTCHA service provided which just uses text image with some less variable font colors and sizes, you can check AZCaptcha service once. I have tried this in past for one of the proof of concepts and it gave quite a satisfactory result. I had to built a VB .NET solution that would make the API call for my free tier account back then which had some call limits per day.

But, also want to let you know that most of these Captcha Solver services which are exposed via API's are governed by one common principle where you send the captcha image via API, the image is queued with a task ID, this task is allocated to a human user at service provider end who will read that image which you sent over and provide the solution as a response that you will be getting back, so there are chances that even they might give you an incorrect response.