Blue Prism Session logs Archival issue - Available storage options

[SrinivasGoripar]

Dear Fellow Members,

I am reaching out to you to get some insights and best practices for handling Session logs / Database archive for Blue Prism Database.

The Archival feature provided in Blue Prism Control room is archiving the information to the local folder space. Even after running the archive from Control room, Session logs size seems not reduced greatly (only archived 100 MB from few GB space in Database). 

Can some one take time to provide help on the below questions please?

1. What are the options available to store it to cloud storage (say Amazon S3 etc.) instead of Local disk. If this is not feasible is there any other effective way to archive?

2. Archival feature seems archived only certain amount of Data from the available logs structure in Control room. The archived folder occupied ~100 MB size on disk. However Database Contains the data in few GB for Session logs.

 Thanks in advance for your time to take a look into my query!

Regards,
Srini

------------------------------
Srinivas Goriparti
------------------------------

[JohnCowell]

One option you have for storing archive files other than a local disk is to copy them across to another destination after the archiving has completed. You could do this in a batch script, for example, and have it run via Windows task scheduler. You could perform the Blue Prism archive via the command line and then on completion, perform the copy of the files over to the new storage destination and then delete the files on the local machine. All this could be done via Windows Task Scheduler batch files.

Archiving works by copying session log data from the database tables based on how the session date is. It's possible that if you see sessions that are never archived then they may not be in a state that archiving considers them for inclusion, such as there is no enddatetime recorded for the session and therefore it could still be running (when in fact it may have terminated and not restarted).

Also consider how SQL Server manages it's space when data is removed. Depending on how you view the database table size, it can appear that after archiving and the sessions are removed from the database the disc space used does not appear to have reduced by much as you would have thought. This is because the space consumed by those sessions has not been given back to the operating system. So a table may appear to consume several gigabytes of space but in fact a lot of that space may be free and ready to be populated again by new data inserts. A good simple way of checking this is to run the Disk Usage by Top Tables report in SQL Server Management Studio : - In SSMS right click on Database, select Reports, Standard Reports, Disk Usage by Top Tables. The report will give you number of rows and kilobytes used per table, including unused.

------------------------------
John Cowell
Senior Software Support Analyst
Blue Prism
------------------------------

[SrinivasGoripar]

Thank you John for your inputs. 

Regarding the freeing up the disk storage space, I ran those reports and Session Logs table still occupy pretty good amount of space in Sql (this has been verified by Top Table Size report)

At the same time, I do not have any more logs to archive or I could not retrieve Logs for a Item Key from Blue Prism Control room. This corresponds to the meaning that all available Session Logs were archived.

But in reality those records still occupy space in the Database. Does the Archive feature available in Blue Prism remove the entries from Database after a successful archive?

When I query the Database (Count(*) from Session log table it shows record count as almost the same. This confuses me if it actually removes the session logs or not.

If not what are the best alternatives available to remove them as a Blue Prism Solutions Developer.

thanks,

------------------------------
Srinivas Goriparti
------------------------------

[JohnCowell]

Hi Srinivas,

Yes, when an archive operation has completed then the archived session data is removed from the database. There is also the option to delete session data without archiving. You may want to log a support ticket with Blue Prism and we can provide housekeeping scripts which can permanently remove old unwanted data from the database. The scripts not only cover session data but also others like work queues. These are useful in some emergency situations where a large amount of data needs to be removed quickly and easily.

You could check the BPASession table with an amended query to see if any rows have a null enddatetime. This means that those sessions did not end gracefully or are still running. See what the startdatetime is for the same row and you will be able to spot those that are very likely old ones which are not running any more.

------------------------------
John Cowell
Senior Software Support Analyst
Blue Prism
------------------------------