Queues Best Practice

[PawelMrozik]

Hello All,

I am currently working on implementing a Blue Prism solution and need a bit more clarity regarding queues. 

Initially I was unsure whether I actually need to use queues at all, but found this thread and I certainly see the value now. 

I then had a dilemma about what exactly to put in the queue. As the solution will process emails, each email is a work item in the queue, with the Outlook Entry ID as the primary key.

I also thought that the queue would not allow duplicate IDs, but that turns out not to be the case, so I had to implement a mechanism to prevent duplicate entries.

Let me know if you might have different approaches or if this seems okay so far.

The question that I have about queues is whether or not it is common for queues to contain different types of work items that could be differentiated via a tag, or are separate queues necessary?

I am leaning toward a single queue unless we are looking at a really high volume of work items, which would not be the case for my project. 

Any hints or suggestions would be greatly appreciated.

------------------------------
Pawel Mrozik
------------------------------

[devneetmohanty07]

Hi Pawel,

One thing that you noted correctly is that the field which you are setting up as the Item Key is not the primary key for the work queues. Every queue in Blue Prism generates an random unique key called as the GUID or the Item ID which actually is the primary key for the work items present in the queue. The Item Key field that we set up is for us to identify any items that is going to get added in the queue. Preferably, you should be using the Item Key field which can be uniquely identified so that it becomes easier for you to filter queues and to get respective items while MI reporting and processing work queue item one after another.

With that being said, the approach you are using is one of the widely followed approaches being followed where while adding items to the queue, we use the action 'Is Item In Queue' and provide the item key value which checks if the item is already present in the queue item with the status as mentioned in the action's input. This action will generate a flag based on which we decide whether to add the item to the queue or skip it entirely.

If you have a huge set of item say tomorrow, the one way to use queues effectively would be to go via Parent-Child relationship concept, where you can have each queue item in parent queue for every date lets say. And each item in the queue which indicates the date will be mapped to the work queue items in the child queue where the item key value is the item key of the parent queue.

So how it can work is, while you add the item to the parent queue lets say A with the date: 07-02-2022 then at the same time you read the outlook emails and add each mail item to child queue let say B with the item key as: '07-02-1996'. You can process the items which have the item key for today's date by applying the key filter value in Get Next Item action and once all the child queue items are either processed or marked as exception, you also mark the parent queue as completed.

This way you can have a very granular control over the queue items and can also view for which date how many items got processed. Your item key can also be the resource name. We implemented a POC where we took a list of resource name and made the process in such a way that only 50 items were added with the Resource A name as Item Key and rest of 100 items were added with Resource B name as Item Key. When we started processing the items one by one we put a condition if the GetResourceName() function value which is an inbuilt Blue Prism function to provide the current running resource name is included in the Item Key then you assign that item to that specific resource. The reason we did this was because in our use case a specific resource had to work on priority items only.


So to answer your question, yes you can extend the queue architecture in multiple ways either by using single or multiple queues. Till you dont have any high volume of data, use the Outlook ID's as Item Key and keep tagging you items with appropriate tag names that can indicate any sub step, or the type of category etc while maintaining the item status as well.

------------------------------
----------------------------------
Hope it helps you out and if my solution resolves your query, then please mark it as the 'Best Answer' so that the others members in the community having similar problem statement can track the answer easily in future

Regards,
Devneet Mohanty
Intelligent Process Automation Consultant | Sr. Consultant - Automation Developer,
Wonderbotz India Pvt. Ltd.
Blue Prism Community MVP | Blue Prism 7x Certified Professional
Website: https://devneet.github.io/
Email: devneetmohanty07@gmail.com

----------------------------------
------------------------------

[PawelMrozik]

Hello Devneet,

First of all, a huge thank you for the thorough reply.

I have a couple of questions.

I do remember the GUID being mentioned in the Blue Prism Foundation Training. So as I understand it, I can set the Key to whatever I want, and I am responsible for checking whether or not the item is already in the queue prior to adding an item. Is the Item ID the same as the GUID? This is from the Is Item In Queue documentation for :

- Item ID

Out

Text

The Item ID of the work queue item

If you have a huge set of item say tomorrow, the one way to use queues effectively would be to go via Parent-Child relationship concept, where you can have each queue item in parent queue for every date lets say. And each item in the queue which indicates the date will be mapped to the work queue items in the child queue where the item key value is the item key of the parent queue.

So how it can work is, while you add the item to the parent queue lets say A with the date: 07-02-2022 then at the same time you read the outlook emails and add each mail item to child queue let say B with the item key as: '07-02-1996'. You can process the items which have the item key for today's date by applying the key filter value in Get Next Item action and once all the child queue items are either processed or marked as exception, you also mark the parent queue as completed.

I am not sure I understand what you mean here. Are you talking about setting up two separate queues, let's say Parent and Child?

So let's say I have five items in the Parent queue:

01-02-2022
03-11-2022
02-01-2022
17-03-2022
03-03-2022

Let's then say that we have a lot of items for today, so 17-03-2022. The child items would be processed for today's date, and close out the 17-03-2022 entry in the Parent queue to mark it as complete?


So to answer your question, yes you can extend the queue architecture in multiple ways either by using single or multiple queues. Till you dont have any high volume of data, use the Outlook ID's as Item Key and keep tagging you items with appropriate tag names that can indicate any sub step, or the type of category etc while maintaining the item status as well.

Thank you for that. To further clarify my question, what I meant is let's say I have a total of four different projects. Each one is a completely different process and they are in no way related. Would you still use the same queue and use tags or create separate queues for each project?




------------------------------
Pawel Mrozik
------------------------------

[devneetmohanty07]

Hi Pawel,

Coming to your first question, the parameter which you can set to uniquely identify the item while adding to the queue and can also use as a parameter in the inputs of the action 'Is Item In Queue' is called as 'Item Key' which may or may not be unique in nature as you are the one defining it, so it depends on your business scenario altogether. Whereas, coming to GUID, it is randomly being generated by Blue Prism every time any item gets added to the work queue. This GUID is also called as 'Item ID' which is returned as an output from the 'Is Item In Queue' action.

Example:

To showcase an example, I took an excel data source where I have set the 'Product Code' column as the 'Key Name' value in the queue settings of my Blue Prism:



Now once the items have been added to the queue, from your Queue Management window, you can see the Item Key that you have used for adding an item as highlighted in red below. However, you can never see the Item ID or GUID from here:



Now if we have a look at the backend database in Blue Prism to check the same. You can see that the column highlighted in red is the Item ID or GUID associated with the item that got randomly generated while the column highlighted in green is the key value column which is set by us while adding the queue items:


So, here you can see that although 'Product Code' is my Item Key, it necessarily is not unique as it totally depends on the data I am providing to Blue Prism work queue. For example, if it were a list of service now incident tickets it might have been unique so it depends on the business data. However, the GUID or the item ID is always going to be unique no matter what as Blue Prism handles that internally.

Coming to your second question now, yes I am talking about two queues here, such that the queue items in child queue are mapped to the individual items in the parent queue. Essentially any parent item would be marked completed only once all the child items in the child queue which are mapped to that parent item in the parent queue item have been processed (been either marked as completed or as an exception). You need to design the process workflow in the similar fashion.

Coming to the last question, you can go ahead with tags and no need to use different queues altogether if the processing steps are not going to be different totally. Tags can help you to filter out the items according to the department that they belong to if you use the department name as one of the tags, it can also help you out to filter out the relevant departments only while generating queue reports to measure the performance of your automation solution.

Multiple queues would only be very  beneficial here if you are going to have lets a number of different processes which may have a completely different steps going to be there which seems to be in your case.

------------------------------
----------------------------------
Hope it helps you out and if my solution resolves your query, then please mark it as the 'Best Answer' so that the others members in the community having similar problem statement can track the answer easily in future

Regards,
Devneet Mohanty
Intelligent Process Automation Consultant | Sr. Consultant - Automation Developer,
Wonderbotz India Pvt. Ltd.
Blue Prism Community MVP | Blue Prism 7x Certified Professional
Website: https://devneet.github.io/
Email: devneetmohanty07@gmail.com

----------------------------------
------------------------------