Data Scraping (Reading Tables from Web Browser especially for Chrome) needs to be improved

[_Amlansahoo]

Data scraping for a web table or a web table which is having multiple pages is very difficult. I know lot of ways are there but there is no convenient way which can be followed for all the browser or web table with multiple pages.

[SHaikh]

Hi Amlan,

I am currently looking for a way to do this , can you please advise how I can collect data from a web table which goes across 12+ pages and store as a collection or text?

Thanks

[_Amlansahoo]

In Blue prism... You can scrap data only from one page at the same.... You have to loop to through all 12 pages and append all the rows into one collection.

Note:- if it is chrome some times it may not work as expected because here we are facing some challenge to read whole table so you may have to loop through each row/cell of the table for each page. Then put them in one collection.

It's not that difficult but very much time taking.

[RusenAktas]

Hi Amlan,

Thank you for submitting your idea. I'm Rusen from the research team. 

Could you please expand on the difficult and/or time consuming parts you've had to deal with?

[_Amlansahoo]

Hi Rusen,

Thanks for your reply.
RIght now issue is if we are retrieving data from a web table and there are 10 pages we have to retrieve from each page then append into a collection. 
Sometimes while reading data from a web table (Especially in Chrome) time out error is appearing. If data scrappin will be a common for all kind of application then it will be much easier to retrieve data and put in a collection.

So I mean to say it should automatically retrieve all the tabled data from all the pages by some additional feature.

Thanks and Regards,
Amlan Sahoo

[HarshitRawat]

I think Amlan is referring to a feature in UIpath called webscrapping which extracts all the data from a webpage acroos multiple webpages

[_Amlansahoo]

Hi Harshit,

You are right. But I am not saying it should be the same kind of ease so that we can use it accross application. Now the problem is for some it's working and for some it's not working for Browser mode it's very weird. Eventually everything we are going to perform on browser mode. Even tomorrow we will Chromium edge, we are gonna use the same. So just tried to write my thoughts.

Thanks

[RusenAktas]

Hi Amlan, Harshit,

Thank you for the explanation. I'm trying to understand how people are using our product for tabled data collection, and what the pain points are. Noted. 

Best,
Rusen

[_Amlansahoo]

Thank you Rusen

[LuukOlthof]

I agree with this Idea. 
Data Scraping should be made easier. We now usually have to figure out ourselves what part of the screen constitutes the table and table rows. Expecially if mutiple pages are involved it takes a lot of working out to get it right. 

What I would like to see is some functionality in the application modeller that recognizes a table and then let's you decide what parts of that table you want to extract. And it should aid in scraping data from multiple pages.