How do the RabbitMQ queues affect the Decipher ML training model?

[dsa206]

Hi,

My question is, how is the RabbitMQ queues related to Decipher ML training model? 

To give context, we trained 600 documents for ML model using RabbitQueue1. Eventually the ML model increased its accuracy.
Using the same ML model(same Batch Type), we wanted to use it for a different queue, say RabbitQueue2.

As for my personal observation, it appears that the ML model we trained for RabbitQueue1 does not take effect in RabbitQueue2. 
Is this normal?

Kind regards,
Dexter Antonio

[Angelo.Carcagno]

Good Morning Dexter,

RabbitMQ is a message queuing system ( aka Message Broker) used for asynchronous communication between different parts of an application. So this is just the go between your applications and how the communicate.

Machine learning models such as Decipher ML are typically trained on specific datasets, and their effectiveness can vary when applied to different datasets. So if you switch the Queue it is new data that it is learning to work with. Which would line up to your tests.

[Tejaskumar_Darji]

As far as I know, ML data is stored in the DB. I don't think it has any correlation with the RabbitMQ queues

Would rely on BP Decipher expert opinion...........

@Ben.Lyons1

[Ben.Lyons1 · Accepted solution]

Hi All,

Tejaskumar is correct. The only data Decipher pushes into RabbitMQ is the completed batch if and the corresponding document ids, no ML or document data is ever sent to, or stored in Rabbit MQ.

The ML capture models are specific to the Document Type they're assigned to, and all documents processed via it will contribute to the model (providing it's configured to train). The only other way documents wouldn't be included is if they're completed and the document training count is not reached by the time the retention period is hit. E.g. 500 documents are completed 1st Sept, the training count is 600 and the retention period is set to 7 days. If the additional 100 documents are processed after the 8th September, the original 500 will have already been deleted and won't be used for training.

Does that answer your question?

Thanks

Ben

[dsa206]

Hi Ben, 

Yes, my question was answered. 

Thanks everyone!

Dex