Digital Exchange

last person joined: an hour ago 

This community is a place to discuss the Blue Prism DX and all of the assets hosted there.
Expand all | Collapse all

Error in Microsoft Azure Computer Vision v3.0

Jump to Best Answer
  • 1.  Error in Microsoft Azure Computer Vision v3.0

    Posted 09-22-2020 07:59

    Hi All,

    While working with asset - Microsoft Azure Computer Vision v3.0
    I am facing one issue in functionality of - Get Read Results.

    I am able to run API to get results in JSON format (Response Content Data variable)
    But the output collection ( Read Results Response ) is coming as blank.
    That means this utility by itself is not converting response to collection and giving a blank collection in results.

    I tried to use BP another asset to convert text to collection , but it is also giving me error

    Any help would be much appreciated!!!




    ------------------------------
    Himanshu Sharma
    Senior Software Engg
    Mercer
    Asia/Delhi
    ------------------------------


  • 2.  RE: Error in Microsoft Azure Computer Vision v3.0
    Best Answer

    Posted 09-25-2020 09:53
    Hi Himanshu,

    I just took a look at the definition of the JSON path query on that action. Right now it's set to:
    $.readResults

    Based on Microsoft's API documentation (Computer Vision 3.0 Reference) I believe it should be:

    $.analyzeResult.readResults​


    I'll have someone take a look at the skill, verify the correct value, and get the skill updated. In the meantime, you can manually change the value in the Web API definition by going to System -> Objects -> Web API Services and editing the action within the Microsoft Computer Vision 3.0 object.

    Cheers,



    ------------------------------
    Eric Wilson
    Director, Partner Integrations for Digital Exchange
    Blue Prism
    ------------------------------



  • 3.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 26 days ago
    @Eric Wilson - Hi, I have question related to Azure Json response.
    Suppose below is my image. After I get a Json response back from Azure (which has everything like Page, text, words, coordinates etc.), I want to get the text from it as displayed in page - left to right. top to bottom
    which will be -

    Text should be extracted from JSON in same way -
    This is my Sample Image
    Key: Testing
    Extract values


    What is the best way to extract the text from JSON response. We probably just need 'Page' (for multipage pdf) and 'text' key from JSON response with Read analyze.
    Will this require a custom C# code and do we have something available in BP for this? ​

    ------------------------------
    Mayank Goyal
    ------------------------------



  • 4.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 26 days ago
    Hi @Mayank Goyal,

    There are a couple ways you could go about extracting specific information.

    1. There's a utility VBO available on the DX (it may be included in the product install as well in the VBO folder) called Utility - JSON​. With this VBO there's an action that will take a JSON blob as input and return it as a parsed Collection. Then you just move through the Collection and pull out what you want.
    2. Another option would be to use a Code stage and the Newtonsoft JSON library to pull the specific information out via a LINQ or JSONP query. This is similar to what the skill is doing to create the Read Results Response collection. Newtonsoft's documentation is pretty good with a decent number of examples. I think the problem you'd face with doing it this way is that if you have multiple Page entries you'll also have to deal with counting them as they're returned as a JSON array.

    Let me know if this helps.

    Cheers,

    ------------------------------
    Eric Wilson
    Director, Partner Integrations for Digital Exchange
    Blue Prism
    ------------------------------



  • 5.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 26 days ago
    @Eric Wilson - Can you help getting a custom C# action created as reusable functionality on DX portal, I am looking for a action were I can provide Json from read/analyze as input and I get as output something - 

    Page​                         Text
    1                        All Text from Page 1 (in sequence - left to right and top to bottom)
    2                        All Text from Page 2 (in sequence - left to right and top to bottom)
    and so on..........

    This might be useful to lot of Blue prism developers.

    ------------------------------
    Mayank Goyal
    ------------------------------



  • 6.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 26 days ago
      |   view attached
    @Mayank Goyal,

    Attached is a quick-and-dirty VBO that can be used to parse the Response Content output of the Get Read Results action. It will return a Collection that contains the page number and complete text found on that page for every page.

    I say this is "quick-and-dirty" because I just threw it together. There is no exception handling in it. So, if you want to use this in production I would encourage you to review the VBO and added any additional exception handling, logging, etc where you see fit.

    Cheers,
    ​​

    ------------------------------
    Eric Wilson
    Director, Partner Integrations for Digital Exchange
    Blue Prism
    ------------------------------



  • 7.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 20 days ago
    @Eric Wilson - Thanks a lot, this was very helpful, will get in touch if stuck with anything else on Azure. ​

    ------------------------------
    Mayank Goyal
    ------------------------------



  • 8.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 13 days ago
    @Eric Wilson - I am getting an error - "Internal : Could not execute code stage because exception thrown by code stage: Data Type mismatch in array" with blue prism VBO Utility Json when converting Json to collection. Please help on the same.
    Its seems related to data type, may be if we can have something that considers everything as String might be useful, however not sure how the code works internally in this VBO. ​​

    ------------------------------
    Mayank Goyal
    ------------------------------



  • 9.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 13 days ago
    Edited by Eric Wilson 13 days ago
    @Mayank Goyal,

    I just went over this with a colleague who experienced the same behavior with another Microsoft service. The issue is that Microsoft are returning mixed numeric types, in some cases, in their JSON arrays. There is a method in the global code of the JSON VBO that checks to determine if the types in an array don't match because that's essentially a bad programming practice.

    Below is an example of how you can change that method in the global code so that an array that has numeric types in it (even if they're different ex. Double vs Int64) will be allowed to pass the check. Just cut-and-paste this code in place of the existing DeserializeArray() action in the Global Code section of the VBO.

    Private Function DeserialiseArray(ByVal o As JArray, ByVal populate As Boolean) As DataTable Dim dt As New DataTable Dim isNumArray As Boolean Dim first As Type = Nothing For Each e As Object In o If first Is Nothing Then first = GetTypeOf(DeserialiseGeneric(e, False)) If (IsNumeric(e)) Then isNumArray = True End If End If If (GetTypeOf(DeserialiseGeneric(e, False)) IsNot first) Then ' Check the array type. If it's numeric we want to make sure the subsequent values are not simply differing number types (ex Int64 vs Double). If (Not isNumArray) Or ((isNumArray) And (Not IsNumeric(e))) Then Throw New Exception("Data Type mismatch in array") End If End If Next If first IsNot Nothing Then dt.Columns.Add(JSON.Array, first) End If If populate Then For Each e As Object In o Dim dr As DataRow = dt.NewRow() dr(JSON.Array) = DeserialiseGeneric(e, True) dt.Rows.Add(dr) Next End If Return dt End Function ​

    We made add this logic to the published VBO at a later date.

      ​

    ------------------------------
    Eric Wilson
    Director, Partner Integrations for Digital Exchange
    Blue Prism
    ------------------------------



  • 10.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 13 days ago
    @Eric Wilson -  ​Before I create a copy of VBO and update this code, i want to understand from you what if we read everything as text (String) from Json and when putting it in collection, with that we will never need to worry about data type?
    Or the fix you have provided already takes care of this? I am not 100% into the code of this VBO at this point of time and only understand it on high level, however want to ensure a permanent fix before I put anything on my environment?

    ------------------------------
    Mayank Goyal
    ------------------------------



  • 11.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 13 days ago
    @Mayank Goyal,

    As I mentioned before, the point of the function​ is to flag any cases where the developer may face an array with mixed data types. Could JSON arrays be treated as string only? Yes, but then in cases where a developer needs numeric values (ex. for calculations) they would then have to cast those values back to numeric data types. That's why we're discussing whether to make a change to that VBO.

    The revised code I posted checks to see what the data type is of the very first item in the array. It then assumes every subsequent item in the array will also be of that same data type. If the first item is a numeric data type, it will verify that every other item is of a numeric data type.

    So, if the first item is a number and the second item is text it will still throw the "Data type mismatch" exception. If the first item is a number (ex. Int64) and the second item is another number, but of a different type (ex. Double), it will not throw the exception.
     
    Cheers,

    ------------------------------
    Eric Wilson
    Director, Partner Integrations for Digital Exchange
    Blue Prism
    ------------------------------



  • 12.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 13 days ago
    @Eric Wilson - Thanks for your input, so if you consider with OCR, the results depend on quality of images and hand writing, so if in a array first item is coming as True and for 2nd item Azure OCR read it as Lroe, we will get error as it wont be a boolean anymore.
    This is why i was thinking with OCR technology, we can have everything as String and developer can handle the logic later to map string to any required datatype, please let me know your thoughts? ​

    ------------------------------
    Mayank Goyal
    ------------------------------



  • 13.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 13 days ago
    @Mayank Goyal,

    I'm not sure that case would apply to OCR because everything returned by Azure Vision, as far as what it's read, is treated as text I believe. The area that we run in to the data type mismatch exception is with their bounding rectangle array definitions in the JSON response, but I understand your point. We'll take it under advisement.

    Cheers,


    ------------------------------
    Eric Wilson
    Director, Partner Integrations for Digital Exchange
    Blue Prism
    ------------------------------



  • 14.  RE: Error in Microsoft Azure Computer Vision v3.0

    Posted 13 days ago
    @Eric Wilson - If you think everything is already read as text, let me go ahead and create a copy of this VBO with updated code ​

    ------------------------------
    Mayank Goyal
    ------------------------------