Decipher Templates - No way to export Exceptions?

[Evert_B_RoboRana]

Hi everyone,

We've build a Blue Prism invoice process using Decipher, where mails in a shared mailbox are read on a daily basis, filtered according to certain parameters, and subsequently the PDF's are sent to Decipher. Next, the blue prism process reads the extracted data and handles the invoice in the invoice booking system. Emails need to be handled throughout the process, so we're using the Process Templates in the Developer Jumpstart section for Decipher to fill a second queue with all this mail data.

If there are exceptions in the PDF files (not invoices), the users operating Decipher can mark these documents as "Exception" and move on. Due to constraints in the way the process is build, we're uploading one PDF file per batch. The issue I'm facing is that the current templates list following as a shortcoming/known constraint:

"There needs to be one response coming from Decipher when Blue Prism polls for Batch Id. The received
batch id is processed further. Without at least one successful document coming from Decipher, the flow
will not proceed. Batches which are completely in exception or deleted do not get processed in
Templates."

Since we're submitting one document per batch, this is a problem because now the Exceptions keep sitting in Decipher instead of being handled by the process, and the users have to manually move the corresponding mails to the correct folder.

Isn't there any way to extract these Exception batches as data in a Blue Prism process?

Many thanks in advance!

[Ben.Lyons1]

Hi Evert,

The concept of an exception in Decipher is slightly different to that in Blue Prism. If a document is an exception, it should mean that the required information cannot be extracted either because the document's incomplete, unreadable or completely the wrong document type (not an exhaustive list). So in terms of the usual exported information defined in the DFD, there's nothing to be exported.

Decipher was designed in such a way that any exception documents do not pass through the data verification and are then downloaded for manual processing.

That being said, there were some updates in v2.1 released earlier this year which affect how exceptions can be configured. What version are you currently using?

What information are you looking to have extracted when no DFD fields can be completed?

Thanks

Ben

[stepher]

Hello Evert,

When you set up the DFD for your project, you set some fields to be 'Required'.  If you remove the 'Required' requirement, so to speak, these documents can pass back into Blue Prism with null/blank values.  You can then evaluate the response within Blue Prism to determine if it was a complete or incomplete return.  The upside of this option is that it alleviates the need of having your end-users or other staff manage Decipher for held documents.  The downside is that Decipher will not improve its reading of the documents without an addtional process.  We are looking at addressing this in a few steps:

1. Either off-cycle from your process run, or after you pause your process
   1. In the configuration screen, set the option to force each document to be manually reviewed in Data Verification
   2. Resubmitting the 'failed' documents under a different Batch Queue Suffix (so they do not get crossed with your base process).
   3. Annotate the 'failed' documents.
   4. Download the batches of 'failed' documents
   5. In the configuration screen, reset the option to allow documents to pass Data Verification
2. Turn your primary process back on, if needed

I am still working on smoothing this process out, but it has worked for us so far.

I hope this helps,
Red

[AndreKöpplin]

Hello Evert

We had simular situation and we solved it in this way:

Add two new lines in our DFD Template "Exception" and "Exception Reason" with below values (Examples):

Exception:
- ID: EXCEPTION_GENERAL
- Flags: Selectable, Required
- ​Option: Yes;No
- Dependent items: Exception Reason
- Default value: No

Exception Reason:
- ID: EXCEPTION_REASON
- Flags: Selectable, SoftValidation
- ​Option: List with possible Exception
- Dependent items: none selected
- Default value: No exception
- Formular: e.g.  IF(EQ(EXCEPTION_GENERAL,"No"),"No exception",SELF)

If Decipher is unable to verfied something, the Business User can check and if it is required, the Business User can select the Exception Reason and pushed (submit) it back to Blue Prism.

You will have two new Document Data outputs from Decipher and you have to adjust your Blue Prism script in this way, that you catch new "Exception" information.

Best regards
Andre

[Evert_B_RoboRana]

Many thanks for your answers so far.

@Ben.Lyons1

In this case, ​Decipher is meant to read invoices coming from a shared mailbox. When the document is not an invoice but a different document type (credit note, reminder, ...), the data verification user can mark it as such (exception), and the process coming after the Decipher stage needs to move this mail toward a certain folder. I suppose I can configure Decipher with a "Document Type" for each type of these exceptions, and have only one or two fields configured in the corresponding DFD. Thoughts on this?

@Robert Stephens

I've tried setting the "not required" fields for these exceptions as Soft Validation fields, but this is not ideal. Furthermore as you indicate, setting these fields as not required has the major downside of not improving Decipher's document reading.

​

[Evert_B_RoboRana]

@AndreKöpplin that actually seems like a good option! However, to submit the batch this means all the necessary fields would have to be put as "Soft​ Validation"? Or do you set all other fields as dependent on this Exception flag?

[Ben.Lyons1]

Hi @Evert Boelaert ​,

Decipher's not able to automatically detect exception reasons, but if (as you proposed) you are able to train a classification model to recognise these alternative documents, you would be able to have them automatically pass through.

You could then have a field with a default value to indicate the document type/outcome/additional Blue Prism action required.

In v2.1 there's a new feature to enable a user to mark a document as an exception, then an exception handler can accept any validations and push it to Blue Prism. All validations become soft validations once marked as an exception. This is configured with a new option box in the Batch Type.

Are either of these options acceptable?

 Thanks

Ben

[AndreKöpplin]

@Evert Boelaert : By all required fields SoftValidation​ have to be selected.