Decipher is Not Training on Invoices

[salman_shaik]

Hi Team,

We have a set of 10 invoices from one vendor we trained the decipher with those invoices by enabling the ML model and keeping Train documents size as 10 and did some changes during manual verification (like vendor name is not correct, tax amount field is not correct, etc..) and submitted the batches. Once the training is completed unchecked the check box Mark for Training in Capture Model and then we have sent the same set of documents to decipher using blueprism.

Even though after training is completed its not recognizing the fields. before the training how it behaved the same thing is happening after training as well then what is the use of training to the decipher.

Are we missing anything during training??

I have sent the 10 documents in so many batches (like 50-60) but still it doing the same thing not reading the values what i teached earlier.

I checked by turning off the ML model by which rules-based training will be enabled automatically but still the same issue is happening.

Another question is when all the documents in the batch are marked as success without any error it's still sitting in data verification even though i have selected all the appropriate setting related to document type and configuration setting.

Auto Skip Data verify.

Skip the Data verification if the accuracy is high.

can you please answer the above questions.

------------------------------
Salman Shaik
------------------------------

[Ben.Lyons1]

Hi Salman,

I would recommend reviewing our best practice guide as you probably don't need to use an ML model and 10 documents is much too low to create a usable model.

Decipher IDP Best Practices

You may need to delete your training data and start from the beginning.

When your documents are marked as successful, but not automatically passing through, is any of the text in the fields red?

Thanks

------------------------------
Ben Lyons
Senior Product Specialist - Decipher
SS&C Blue Prism
UK based
------------------------------

[salman_shaik]

Hi @Ben.Lyons1,

Yes, I delete the entire training data and started from beginning and it contains some text in the fields which are in red colour even though the value is correct. So, i selected the region again for the particular field but still the text is showing in red colour only. what is the reason behind that & how to eliminate the issue?

Apart from the above i have the below questions.

1. As we are getting invoices from different vendors the date format is not fixed like it may contain (dd/mm/yyyy or like 28-Aug-2022) but we want the date should be in mm/dd/yyyy format only for further processing. we can't throw it as an exception because its only format related issue. how to handle this inside decipher while extracting the date without going to the process level?
2. While training the documents we verified the batch and submitted it. we created the pre-defined exception for batch type & document type, after that we sent a batch type which contains a mandatory field is missing inside some documents then the decipher divided the batches into two one is sent to RabbitMQ and other one is sent for Manual verification. But inside manual verification it should create an exception tab and send the documents to exception tab but the decipher is not doing that. is there any issue?
3. Can we use Exclude keywords & Selectable both at a time for a single header in DFD? in our case it's not working.
4. Let's say we have some training data which contains 10 invoices, and a mandatory field is missing in all these invoices. but now we want to train the decipher with these 10 invoices by neglecting the mandatory field by manually entering the data in the field during manual verification. when we move this process to prod it should check for that mandatory field however, we entered the value manually during training obviously it should fail. Is there any way to achieve this use case instead of unchecking the flag Required in DFD?

Please address the above issues....

Thanks,

------------------------------
Salman Shaik
------------------------------

[Tejaskumar_Darji]

If the default date conversion is not working then you can handle this by keeping the date format as Original in the Batch Types settings

and then in BP handle all date conversion scenarios. This way at least Decipher will not throw exception and let is pass through

------------------------------
If I was of assistance, please vote for it to be the "Best Answer".

Thanks & Regards,
Tejaskumar Darji - https://www.linkedin.com/in/tejaskumardarji/
Technical Lead
------------------------------

[Ben.Lyons1]

Hi Salman,

The red/brown text (version 2.2+ can contain both, 2.1 is red only) indicates low confidence, red is low OCR confidence and brown is low field confidence. For OCR confidence to be high, it needs to be 86%+, field confidence is only calculated for high OCR confidence fields. High field confidence is 95%+.

Unfortunately this information is not readily available (I'm working on it), but you can use the CCL/FCL misc parameters to lower the respective confidence threshold e.g. CCL = 70 would lower the OCR confidence threshold to 70%. Some trial and error will be involved to find a suitable value.

I've seen this issue with some dates myself, and I think my early testing of 2.3 has should some improvement (currently due Q4, pending successful testing).

1. What language/locale setting do you have in your Document Type and Batch Type? Having US and UK will conflict with one another.
2. I'm not sure this is what you're suggesting, but Decipher won't automatically mark a document as an exception, this needs to be done manually. Then to view the exceptions tab, the user/user group needs to have the "ResolveExceptions" permission added.
3. I don't see why not, those features are not connected. If you wish to exclude a keyword from being used to identify a field and use the 'Selectable' flag to set the field as a dropdown, this should work. Assuming they do not contradict/conflict with one another and you've specified a list of selectable values in Options or as a List.
4. I wouldn't make any DFD changes in production, as this may affect the success rate achieved in testing. Depending on the reason for having a blank mandatory field, you could either set it to "SoftValidation" (flag), use the "DisableSoftValidation" misc parameter to be more specific, or perhaps include a different setting for the specific vendor as a Specific Version.

You may find it useful to request a knowledge sharing session via our customer support portal.

Thanks

------------------------------
Ben Lyons
Senior Product Specialist - Decipher
SS&C Blue Prism
UK based
------------------------------