Decipher going to Manual Verification stage

[AvinaSemlani]

Hi,

I am currently training Decipher to read different types of invoices. I have set the confidence threshold as 95%. In certain situations, Decipher is going to Manual Verification stage even when it has read all the fields accurately. I also noticed it is highlighting the quantity column as RED even when it has read it correctly. Is this the reason its going for Manual verification?
How do we make sure that it does not go to Manual Verification in the future once it has read all the fields correctly?

#Decipher #IDP #ManualVerification​​​​

------------------------------
Avina Semlani
ASE
Accenture
Asia/Kolkata
------------------------------

[HarpreetKaur]

Hello Avina,
Are you talking about the Classification confidence threshold field in Document Type?
That percentage is for Document Classification not Data Verification. The percentage threshold check kicks in when you set your Classification to automatic under Config.

As far as the data extraction goes, the manual verification step is introduced to indicate the docs that Decipher has trouble reading. Decipher might be able to extract the info correctly but it does not do so with a high confidence and hence marks those fields as warning in Red.
You can handle that by either introducing more specific formulas or regex for those particular fields in your DFD or try to analyze the document patterns that regularly get spit out as exceptions. you can then gather more documents of similar structure (minimum 50)to re-train Decipher on them..
Ultimately with machine learning, it is hard to explain with what precise no. of docs would Decipher start hitting that threshold. sometimes works with 10, but at times even 100 docs aren't enough.
You can also set these fields as soft warning in Decipher which will still enable Decipher to extract these values and these can further be verified in your BluePrism solution using certain business rules.

------------------------------
Harpreet Kaur Product Consultant
------------------------------

[BenLyons]

Hi Avina,

If you're happy with your document training and ready for enabling auto-verification, you can take the following steps:

1. Go the admin panel
2. The click on Configuration under the System heading
3. Switch on the auto-skip for the respective modules

This will be enabled for all batches, so be mindful that this will also affect documents till being trained. The way to manage that is to ensure you have added the "Required" flag to all essential fields.

If they are still being held up, please get in touch.

Thanks

------------------------------
Ben Lyons
Product Consultant
Blue Prism
UK
------------------------------

[AvinaSemlani]

Hi Harpreet,

Thankyou for the clarification on Classification confidence threshold.

As per your suggestion, I analyzed the document patterns that gave the RED error. I realized that it does so when the quantity field has special characters in it like "3.0". I changed it to text and I am not getting the error anymore. 

Also, can you let me know how to set soft warning in Decipher?

Appreciate your support.
Thanks

------------------------------
Avina Semlani
ASE
Accenture
Asia/Kolkata
------------------------------

[AvinaSemlani]

Hi Ben,

Thankyou for your response.

I have applied the settings you suggested.

From what I understand, once auto-verification is enabled, Decipher will not got to Manual Verification stage once it reads all the fields that are marked as Required. Please let me know if my understanding is correct.

Thanks

------------------------------
Avina Semlani
SSE
Accenture
Asia/Kolkata
------------------------------

[BenLyons]

Hi Avina,

That's pretty much correct, the only additional reason(s) a document may get held up is if one of the fields fails a validation criteria.

For example, Decipher has read a field that includes a formula for validation, but the field does not meet the criteria of the formula. This might be where you have a table with a value column, then a total field. The total field could include a formula to check the total field is the sum of the table column. If they don't match, the document will be held for manual review.

There's a flag in the DFD where you can set it as a Soft Validation, this will allow you to accept the warning manually. Without the flag, the warning will have to be resolved so the validation is correct.

Thanks

------------------------------
Ben Lyons
Product Consultant
Blue Prism
UK
------------------------------

[AvinaSemlani]

Thankyou for the response Ben. 

So if I set the flag as soft validation for a particular field, that document will not go to the Data Verification stage even if there's a warning? And once it is processed by decipher, the status will be Verify performed - ready to export? 


Thanks

------------------------------
Avina Semlani
SSE
Accenture
Asia/Kolkata
------------------------------

[BenLyons]

Hi Avina,

I think a soft validation won't enable it to pass through without manual confirmation. It's just that it will raise it as an issue, which can be manually accepted. Without the flag, it cannot be manually accepted.

Thanks

------------------------------
Ben Lyons
Product Consultant
Blue Prism
UK
------------------------------