topic RE: BlueBrism - Web Scrapping page by page in Product Forum

BlueBrism - Web Scrapping page by page

FayazAli — Fri, 06 Sep 2019 05:48:00 GMT

Scenario : Do web scrapping page by page[ Assume my application may have pagination with table data ]

Challenges :

1. I am able to add table data into collection using Read stage ---> inside, GetTable option into Collection Data Item.
2. Now, i want to do the same for all pages navigating page by page.
3. I have no idea which option i suppose to use as i am begginer.
4. I tried whether i can use while loop assuming untill i have Next button is visible=TRUE.

But, none of the options are working fine. Kindly any experts help me. Thanks in advance.

------------------------------
Fayaz Ali
RPA Developer
Accenture
UTC
------------------------------

RE: BlueBrism - Web Scrapping page by page

GopalBhaire — Fri, 06 Sep 2019 10:45:00 GMT

1. Use a wait stage to check if the Next button exist
2. If yes it'll press the button -> wait for page load -> read table and go back to step 1
3. If not then end

Assuming Next button is not present on last page, even if it's present it'll be deactivated check for that attribute and spy accordingly.

------------------------------
Gopal Bhaire
Analyst
Accenture
------------------------------

RE: BlueBrism - Web Scrapping page by page

FayazAli — Sat, 07 Sep 2019 13:13:00 GMT

Hi @GopalBhaire bro, thanks a lot for your time and reply. Yes, it worked based on your suggestions. But, i am facing different issue now.

I am able to navigate page by page and extract the table content into collection, but every time i put my table data into collection, it just takes for that particular page data, it is not appending to collection. Pls guide me did i missing anything here?

attached the image.

------------------------------
Fayaz Ali
RPA Developer
Accenture
UTC
------------------------------

RE: BlueBrism - Web Scrapping page by page

joakimeklund — Sun, 08 Sep 2019 16:49:00 GMT

Hi,

Create two Collections (Collection1 and Collection2)
With Read Stage output the table into Collection1
Use the Collection Manipulation VBO (can be found within the Blue Prism directory and then imported)
Use the Append collections action with Collection1 and Collection2, output to Collection2

/Joakim

------------------------------
Joakim Eklund
Senior RPA Developer
Swedbank AB
Europe/Stockholm
------------------------------

RE: BlueBrism - Web Scrapping page by page

FayazAli — Mon, 09 Sep 2019 09:59:00 GMT

Thanks bro, it worked.

------------------------------
Fayaz Ali
RPA Developer
Accenture
UTC
------------------------------