topic Re: Decipher Capture Model Not Training in Product Forum

Decipher Capture Model Not Training

AndersonCachiol — Thu, 04 Nov 2021 17:48:32 GMT

Hello!

I'm trying to train the capture model but it is not working. How long does it take to train? I've been trying since yesterday and it still did not updated the training date.

Re: Decipher Capture Model Not Training

Ben.Lyons1 — Fri, 05 Nov 2021 08:05:49 GMT

Hi Anderson,

Have you followed the installation steps to enable the ML training?

Also, looking at the document count you're looking to train, you may need to increase the document count. If this is just a test, then no problem, you should be able to see how the training works. However, training based on 10 documents will give you a very low quality model and will likely have little positive effect on the outcome.

When you're looking to create something for production, I would recommend taking a look at our best practice guide which covers the training document count topic.

Thanks

Ben

Re: Decipher Capture Model Not Training

AndersonCachiol — Fri, 05 Nov 2021 11:18:38 GMT

Hello Ben!

Yes, I have enabled the ML training.

The idea of having 10 documents as universe, was just to test if the ML is making any progress (even if minimal)

The configuration we intent to use for production would be 500 docs as training size and periodic of every 20 (because we have lots of different layouts that need to be updated overtime)

I've noticed that the "Last trained" field is still N/A (and I've added a few documents after the ss above), but yesterday some of the documents have showed a good capture in one of the batches, and then on the next one it was really bad... is this expected?

Sorry if I'm asking too much stuff, but it is my first time working with Decipher (with any OCR platform, to be honest)

I've completed the Decipher foundation training but I still have doubts and I don't know if there's some place else to ask for help.

It would be really good to have a talk over skype / teams / zoom with someone who understands decipher really well, to get some orientations and clarifications, is this possible?

Re: Decipher Capture Model Not Training

Ben.Lyons1 — Fri, 05 Nov 2021 11:35:24 GMT

Hi Anderson,

I understand, it's a good idea to check how it works before you put 500 documents through it.

Ok, there's chance that the training isn't happening because the services have started in the wrong order. I think you can resolve this by stopping the Decipher Server service and the Decipher Client service, then restart them. The client service must be started last. You may see the document training count temporarily drop to zero whole it resets and if the cached documents have expired (based on the data retention rules), they may not return. In this event you'll need to resubmit them.

In addition there are some troubleshooting steps in the help pages under "Machine learning training doesn't seem to be working".

With respect to training count, it's unlikely you'll see any difference using a model based on 10 or 20 documents. As per the best practice guidance, I would recommend setting this to between 5 - 10 times the number of different layouts (depending on the document complexity). So if you have 100 layouts, you would have a minimum of 500 to train the initial model.

Then for retraining, you don't want this to happen constantly as it will use up a lot of CPU and slow down your document processing. I would recommend setting it at around your expected daily/weekly volume, depending on your use case size.

A capture model is there to enhance an already well trained DFD, so ensure you've followed the best practice steps before enabling this in production.

Thanks

Ben

Re: Decipher Capture Model Not Training

AndersonCachiol — Fri, 05 Nov 2021 12:20:04 GMT

Ben,

I've checked the troubleshooting page as you mentioned and found that the Data Capture Client log does not have the statements of ML initialization.

I've already restarted the services in the order you said and now it is initializing the ML. I think this will probably solve the problem.

Thanks 😄