BlueBrism - Web Scrapping page by page

[FayazAli]

Scenario : Do web scrapping page by page[ Assume my application may have pagination with table data ]

Challenges :

1. I am able to add table data into collection using Read stage ---> inside, GetTable option into Collection Data Item.
2. Now, i want to do the same for all pages navigating page by page.
3. I have no idea which option i suppose to use as i am begginer.
4. I tried whether i can use while loop assuming untill i have Next button is visible=TRUE.

But, none of the options are working fine. Kindly any experts help me. Thanks in advance.

------------------------------
Fayaz Ali
RPA Developer
Accenture
UTC
------------------------------

[GopalBhaire]

1. Use a wait stage to check if the Next button exist
2. If yes it'll press the button -> wait for page load -> read table and go back to step 1
3. If not then end

Assuming Next button is not present on last page, even if it's present it'll be deactivated check for that attribute and spy accordingly.

------------------------------
Gopal Bhaire
Analyst
Accenture
------------------------------

[FayazAli]

Hi @GopalBhaire bro, thanks a lot for your time and reply. Yes, it worked based on your suggestions. But, i am facing different issue now.

I am able to navigate page by page and extract the table content into collection, but every time i put my table data into collection, it just takes for that particular page data, it is not appending to collection. Pls guide me did i missing anything here?

attached the image.​

------------------------------
Fayaz Ali
RPA Developer
Accenture
UTC
------------------------------

[JoakimEklund · Accepted solution]

Hi,

Create two Collections (Collection1 and Collection2)
With Read Stage output the table into Collection1
Use the Collection Manipulation VBO (can be found within the Blue Prism directory and then imported)
Use the Append collections action with Collection1 and Collection2, output to Collection2

/Joakim

------------------------------
Joakim Eklund
Senior RPA Developer
Swedbank AB
Europe/Stockholm
------------------------------

[FayazAli]

Thanks bro, it worked.

------------------------------
Fayaz Ali
RPA Developer
Accenture
UTC
------------------------------