Error in Microsoft Azure Computer Vision v3.0

[himanshu__sharma]

Hi All,

While working with asset - Microsoft Azure Computer Vision v3.0
I am facing one issue in functionality of - Get Read Results.

I am able to run API to get results in JSON format (Response Content Data variable)
But the output collection ( Read Results Response ) is coming as blank.
That means this utility by itself is not converting response to collection and giving a blank collection in results.

I tried to use BP another asset to convert text to collection , but it is also giving me error

Any help would be much appreciated!!!

------------------------------
Himanshu Sharma
Senior Software Engg
Mercer
Asia/Delhi
------------------------------

[ewilson · Accepted solution]

Hi Himanshu,

I just took a look at the definition of the JSON path query on that action. Right now it's set to:
$.readResults

Based on Microsoft's API documentation (Computer Vision 3.0 Reference) I believe it should be:

$.analyzeResult.readResults​

I'll have someone take a look at the skill, verify the correct value, and get the skill updated. In the meantime, you can manually change the value in the Web API definition by going to System -> Objects -> Web API Services and editing the action within the Microsoft Computer Vision 3.0 object.

Cheers,

------------------------------
Eric Wilson
Director, Partner Integrations for Digital Exchange
Blue Prism
------------------------------

[MayankGoyal2]

@ewilson - Hi, I have question related to Azure Json response.
Suppose below is my image. After I get a Json response back from Azure (which has everything like Page, text, words, coordinates etc.), I want to get the text from it as displayed in page - left to right. top to bottom
which will be -

Text should be extracted from JSON in same way -
This is my Sample Image
Key: Testing
Extract values

What is the best way to extract the text from JSON response. We probably just need 'Page' (for multipage pdf) and 'text' key from JSON response with Read analyze.
Will this require a custom C# code and do we have something available in BP for this? ​

------------------------------
Mayank Goyal
------------------------------

[ewilson]

Hi @Mayank Goyal,

There are a couple ways you could go about extracting specific information.

1. There's a utility VBO available on the DX (it may be included in the product install as well in the VBO folder) called Utility - JSON​. With this VBO there's an action that will take a JSON blob as input and return it as a parsed Collection. Then you just move through the Collection and pull out what you want.
2. Another option would be to use a Code stage and the Newtonsoft JSON library to pull the specific information out via a LINQ or JSONP query. This is similar to what the skill is doing to create the Read Results Response collection. Newtonsoft's documentation is pretty good with a decent number of examples. I think the problem you'd face with doing it this way is that if you have multiple Page entries you'll also have to deal with counting them as they're returned as a JSON array.

Let me know if this helps.

Cheers,

------------------------------
Eric Wilson
Director, Partner Integrations for Digital Exchange
Blue Prism
------------------------------

[MayankGoyal2]

@ewilson - Can you help getting a custom C# action created as reusable functionality on DX portal, I am looking for a action were I can provide Json from read/analyze as input and I get as output something - 

Page​                         Text
1                        All Text from Page 1 (in sequence - left to right and top to bottom)
2                        All Text from Page 2 (in sequence - left to right and top to bottom)
and so on..........

This might be useful to lot of Blue prism developers.

------------------------------
Mayank Goyal
------------------------------

[ewilson]

@Mayank Goyal,

Attached is a quick-and-dirty VBO that can be used to parse the Response Content output of the Get Read Results action. It will return a Collection that contains the page number and complete text found on that page for every page.

I say this is "quick-and-dirty" because I just threw it together. There is no exception handling in it. So, if you want to use this in production I would encourage you to review the VBO and added any additional exception handling, logging, etc where you see fit.

Cheers,
​​

------------------------------
Eric Wilson
Director, Partner Integrations for Digital Exchange
Blue Prism
------------------------------

[MayankGoyal2]

@ewilson - Thanks a lot, this was very helpful, will get in touch if stuck with anything else on Azure. ​

------------------------------
Mayank Goyal
------------------------------

[MayankGoyal2]

@ewilson - I am getting an error - "Internal : Could not execute code stage because exception thrown by code stage: Data Type mismatch in array" with blue prism VBO Utility Json when converting Json to collection. Please help on the same.
Its seems related to data type, may be if we can have something that considers everything as String might be useful, however not sure how the code works internally in this VBO. ​​

------------------------------
Mayank Goyal
------------------------------

[ewilson]

@Mayank Goyal,

I just went over this with a colleague who experienced the same behavior with another Microsoft service. The issue is that Microsoft are returning mixed numeric types, in some cases, in their JSON arrays. There is a method in the global code of the JSON VBO that checks to determine if the types in an array don't match because that's essentially a bad programming practice.

Below is an example of how you can change that method in the global code so that an array that has numeric types in it (even if they're different ex. Double vs Int64) will be allowed to pass the check. Just cut-and-paste this code in place of the existing DeserializeArray() action in the Global Code section of the VBO.

Private Function DeserialiseArray(ByVal o As JArray, ByVal populate As Boolean) As DataTable Dim dt As New DataTable Dim isNumArray As Boolean Dim first As Type = Nothing For Each e As Object In o If first Is Nothing Then first = GetTypeOf(DeserialiseGeneric(e, False)) If (IsNumeric(e)) Then isNumArray = True End If End If If (GetTypeOf(DeserialiseGeneric(e, False)) IsNot first) Then ' Check the array type. If it's numeric we want to make sure the subsequent values are not simply differing number types (ex Int64 vs Double). If (Not isNumArray) Or ((isNumArray) And (Not IsNumeric(e))) Then Throw New Exception("Data Type mismatch in array") End If End If Next If first IsNot Nothing Then dt.Columns.Add(JSON.Array, first) End If If populate Then For Each e As Object In o Dim dr As DataRow = dt.NewRow() dr(JSON.Array) = DeserialiseGeneric(e, True) dt.Rows.Add(dr) Next End If Return dt End Function ​

We made add this logic to the published VBO at a later date.

  ​

------------------------------
Eric Wilson
Director, Partner Integrations for Digital Exchange
Blue Prism
------------------------------

[MayankGoyal2]

@ewilson -  ​Before I create a copy of VBO and update this code, i want to understand from you what if we read everything as text (String) from Json and when putting it in collection, with that we will never need to worry about data type?
Or the fix you have provided already takes care of this? I am not 100% into the code of this VBO at this point of time and only understand it on high level, however want to ensure a permanent fix before I put anything on my environment?

------------------------------
Mayank Goyal
------------------------------