Work Queues - Storing Data As Binary Data Item - Best Practice?

[IanCampbell]

Hi,

I currently have a process that requires multiple steps across different applications (basically download, variations of data manipulation and then upload) and due to certain restrictions on data storage and to allow running different parts of the process across multiple resource machines, the decision was taken to keep the files from the different stages of the process in the queue item being worked.

I have seen the occasional comment on here and elsewhere stating that this is not best practice. However, I have yet to find an explanation in any documentation/guides that specifically state this. The reason I'm looking in to this is that it appears that when storing/retrieving the data from the queue item, the local BP loses connection to the database (still retrieves the data but shows 'Warning: Failed to Update Database Timestamp' in the local BP client).

Does anyone have any information on:
1) Whether the size of the files (added as binary data, up to 5MB maximum) is causing an overload in the local client and preventing the update to the database.
2) Best practice around storing of data in work queue items. It would be a useful solution if this worked as then it would only require the maintenance of the BP database and queue items (each of which contained their own data history for use in audits etc).

For reference, we are currently using BP v6.2.1

I look forward to hearing any thoughts around this.

------------------------------
Ian Campbell
RPA Specialist
Allen & Overy
------------------------------

[John__Carter]

Hi Ian I think the caution advised when taking this approach is around the risk bloating the DB. I can recall using a queue for screenshots that seemed like a great idea at the time until months later when the DBA was asking why our DB was so huge - we'd stupidly forgotten about maintaining the screenshot queue. So the advice isn't 'don't ever do that', but more be careful/pragmatic/realistic, design maintenance into your solution, and don't treat the BP DB as some sort of universal/eternal data repository.

I'm not aware of a specific size limit, but it stands to reason that the bigger the file the more effort to transport and store it. Might be worth asking the question to the Support team.

What about a file share location instead?​

------------------------------
John Carter
Professional Services
Blue Prism
------------------------------