Getting Started

 View Only
last person joined: 3 days ago 

If you're new to Blue Prism, we'll help you get started.

Expand all | Collapse all

AWS S3 API Calls fails when objects has special characters or spaces in the name

  • 1.  AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 14:26

    Greetings,

    I am using the last AWS S3 object from DXchange and all the prequisite objects as well are up to date.

    I managed to use the main actions such as uploading, downlading and so. However, they only work if the object name has no space or special characters.
    According to AWS documentation, this scenario needs a special handling. It makes complete sense to do so as it will be affecting the URL and it has to be encoded. 


    I am not good with custom codes, but I understood that there is no encoding happening. which leads to the problem I'm facing.
    Is there any possibility to get any kind of help to modify the code to make it encode the special characters?

    I've been trying to solve that issue for over than a week. I tried to build my own webservice action but I got stuck with AWS Signature.

    Main goal is to download files which can be achieved by GetObject action.

    I would appreciate any kind of help. Thanks in advance!



    ------------------------------
    Eslam Ghandour
    ------------------------------


  • 2.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 18:52
    Hi @Eslam Ghandour,

    When you say "object name" you're referring to the bucket name, correct? ​If so, it's pretty easy to URL encode the data before you pass it into the specific S3 action on the VBO. There's another VBO available on the DX called Utility - HTTP. You may already have it installed. If so, check to see if you have the version that includes the action titled URL Encode. If you do, you can pass your bucket name into that action and it will return the URL encoded version of the string. Then you can pass that into the specific S3 action.

    If you don't have a version of the Utility - HTTP VBO with the URL Encode action, you can download the latest version here.

    Cheers,

    ------------------------------
    Eric Wilson
    Director, Integrations and Enablement
    Blue Prism Digital Exchange
    ------------------------------



  • 3.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 19:15

    Hi @Eric Wilson,

    Thank your taking the time to help!

    By "object name" I meant the file that I need to download from S3. As you are probably I ware, I need to pass the bucket name and the key of the file that needs to be downloaded. The bucket has no special characters, the file names do.

    Considering this example /Bucket-Name/FirstFolder/SecondFolder/File to download 10:20:30.pdf

    I did try using the URL Encode action​ in the Utility - HTTP on FirstFolder/SecondFolder/File to download 10:20:30.pdf since it's the key for the object to be downloaded. However, this still did not work.
    It also encoded the "/" which doesn't seea problem for the request either ways as I can download without issues if the name of the file as FileToDownload (with no special characters)

    Do you think this is the correct approach to change only that or are you suggesting using it on the full AWSEndpoint?
    I tried to use it on the full AWSEndpoint and I received an error  when I passed it to the HTTP Request File action.



    Looking forward for your answer.



    ------------------------------
    Eslam Ghandour
    ------------------------------



  • 4.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 19:56
    @Eslam Ghandour,

    If you open the Utility - HTTP VBO and go to the URL Encode action you'll find a single Code stage on the page. Go into the actual code of that stage and change the following line of code:

    Encoded_URL = HttpUtility.UrlEncode( URL )​


    to this

    Encoded_URL = HttpUtility.UrlPathEncode( URL )


    This will encode the file name, but it will leave the "/" untouched.


    Cheers,

    ------------------------------
    Eric Wilson
    Director, Integrations and Enablement
    Blue Prism Digital Exchange
    ------------------------------



  • 5.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 20:21
    Hi @Eric Wilson,

    Thank you very much for your fast replies and your support. I really appreciate that.

    On the other hand, as this is for sure a very valuable information and it did achieve the task of encoding only the path, sadly it did not solve the original issue.

    I'm still facing the main issue which is the http request still failing to go through. It appears to me that it is something within the code stage in HTTP Request File action.

    I'm facing a dead-end here and I can't think of any other reason.

    I am sure that the problem is related to the key being passed as it works fine when there are no special characters in it. Additionally, I tried the same request with the key that has special characters using postman and it works fine as well. Taking in consideration that it works with postman's automatic encoding and when I disable it and encode the special characters manually in the URL.

    It appears to me that I know the problem and why it's failing but I have no clue on how to solve it.

    I am sorry for any inconvenience and I just want to thank you so much for helping!

    ------------------------------
    Eslam Ghandour
    ------------------------------



  • 6.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 21:28
    @Eslam Ghandour,

    Ok, so you're able to execute the actions (ex. GetObject) and everything works fine as long as the specific object doesn't include spaces or other special characters in the name. As soon as the object includes spaces, the request fails. Is that an accurate description of the issue? When the request fails, do you receive a specific error response message?

    Cheers,


    ------------------------------
    Eric Wilson
    Director, Integrations and Enablement
    Blue Prism Digital Exchange
    ------------------------------



  • 7.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 21:36

    @Eric Wilson​,

    Yes, this is an accurate description. 

    The error I receive is a 403 forbidden.

    At first I thought it might be related to permissions but that was cleared out as I already can download other files using the same GetObject action. As well as I can download the those files with space using WinScp.



    ------------------------------
    Eslam Ghandour
    ------------------------------



  • 8.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-27-2022 23:30
    @Eslam Ghandour,

    Ok, I've figured out what the issue is. Deep inside the VBO title BluePrism AWS REST API ​Utility there is a line that performs URL encoding of the request path. The problem appears to be that this is causing AWS to view the calculated signature on the request as invalid. I'm trying to understand why this step was added before I make any changes though. Stay tuned.

    Cheers,

    ------------------------------
    Eric Wilson
    Director, Integrations and Enablement
    Blue Prism Digital Exchange
    ------------------------------



  • 9.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name
    Best Answer

    Posted 09-28-2022 03:12
    @Eslam Ghandour,

    We just pushed version 2.2 of the BluePrism AWS REST API Utility VBO to the DX.​ This addresses the issue you are seeing with files that include spaces in their name. It turns out there was a double URL encoding scenario going on.

    Cheers,

    ------------------------------
    Eric Wilson
    Director, Integrations and Enablement
    Blue Prism Digital Exchange
    ------------------------------



  • 10.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-28-2022 07:44
    Hi @Eric Wilson.

    This is great, it did indeed solve the space issue. Thank you very much for your input and the release!

    However, I am still facing the same issue due to the colon. Taking the same previous example FirstFolder/SecondFolder/File to download 10:20:30.pdf it is still failing because of ":"
    I tested the new version with multiple scenarios and the space for sure is working and I tested it not only in GetObject but also PutObject.
    My assumption is that the characters mentioned in AWS documentation (screenshot in original message) that needs special handling are not being specially handled. The colon is one of them.

    I don't really understand what AWS means by special handling. Do you think this is something  that can be handled in a way on process level or you will have to find a solution and make another version of it?

    Looking forward for your reply and thanks again for the great release!

    ------------------------------
    Eslam Ghandour
    ------------------------------



  • 11.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-28-2022 15:00
    Hi @Eslam Ghandour,

    The colon (:) is not considered a valid character for use in filenames when it comes to Windows. In fact, if you try to create a new file on your Windows machine and use a colon in the name you should see something like this:

    While the colon can be used in filenames on *nix systems (i.e. Unix and Linux) it's generally frowned upon because files can't be moved between *nix and Windows and because the colon can be a special character in various scripting languages. Further, the colon is considered a reserved character from the perspective of RFC 3986 and URIs.

    My recommendation is, if possible, don't use colons, or any other reserved character, in your filenames.

    Cheers,

    ------------------------------
    Eric Wilson
    Director, Integrations and Enablement
    Blue Prism Digital Exchange
    ------------------------------



  • 12.  RE: AWS S3 API Calls fails when objects has special characters or spaces in the name

    Posted 09-28-2022 18:24
    @Eric Wilson

    Thank you very much for your brief explanation and the fast release to solve that issue. Your effort is really appreciate!

    I will follow your recommendations.

    Best of luck! ​

    ------------------------------
    Eslam Ghandour
    ------------------------------



Welcome to Getting Started Community!

Here is a the best place to start your Blue Prism software journey. Watch our guided tour of the Blue Prism ecosystem, and download your handy playboard that will guide you through everything you need to know to be set up for success. Are you brand new to intelligent automation? Here you can get a helping hand with any questions you might have.

This Global Community is intended to help new users get started with Blue Prism products & services. For more specific technical communities please visit this page.

Online DocsJoin our Mentoring ProgramBlue Prism CloudROM EssentialsGet Started with Blue Prism Methodology

FAQs

Stay informed on the latest product updates, webinars and user events we host around the world via our monthly newsletter.

Sign up for the newsletter here

Subscribe to product announcements here

The ROM provides the foundations to embrace Digital Workers and establish a seamless bi-directional flow of work between the human and the digital workforce, starting with vision and strategy. It has been developed using more than a decade of experience and knowledge of deploying Blue Prism at scale in countries and industries across the globe. With the ROM in your corner, you can approach and successfully deliver maximum business benefit through the scaled deployment of connected-RPA. Explore the ROM now.

Get up to speed quickly, enhance your product knowledge and become a certified RPA practitioner through the Blue Prism University. We offer world class training for our customers so you can gain the skills and the confidence you need to successfully deliver outstanding solutions with Blue Prism and enable your team to do the same. Explore the University now.

You can find easy to follow instructions for all the tools and features of the Blue Prism product. This is your one-stop-shop for all the latest product news, release notes, installation guides, how-to instructions, product guides and more. Explore Online Help now.

If you don’t find your answer on the Help System, you can search our extensive knowledge base to find answers to frequently asked questions or browse featured articles covering key aspects of the product. Within a few clicks you can access information to help you troubleshoot, guide you through installation, design processes, or to upgrade and expand your Digital Workforce. And, if you don’t find your answer in the Knowledge Base, you can always raise a ticket* with our Support Center.

The Success Accelerator program combines various levels of mentorship and access to our Expert Services, Technology Ecosystem and Certified Partners based on the size and maturity of your digital workforce operations. For customers getting started, we have designed a Starter Package to assure you’re off to a perfect start. Among other services, you’ll gain access to the myriad of connected-RPA technologies, a ROM assessment, mentoring by Blue Prism experts, and guidance as you identify and build your first automation. Explore the Success Accelerator now.

The community is an exciting platform where you can engage with other Blue Prism users who share common interests, goals and objectives. Whether you're someone looking to engage in open and innovative conversation, a customer wishing to be a source of input for upcoming products, or a developer looking to share best practices, knowledge and insight, the Blue Prism Community is where to do it. Explore the community now.

The Process Discovery Tool is a free-to-use online tool that assesses and ranks your processes in terms of most automation-ready to least, with insight into the potential financial impact. You can access the tool on the Customer Portal and by answering a few simple questions, you’ll be able to map your processes into four categories and then prioritize them accordingly. The tool provides your team with a user-friendly results dashboard that centralizes all discovery activity, with real-time information on process discovery progress that can be effectively and impact-fully communicated to senior management. Explore the tool now.

We have a variety of customer case studies highlighting customer challenges, solutions and the success they’ve had using intelligent automation to drive value and deliver a host of benefits to their businesses. We hope you’ll be inspired by hearing how other organizations are successfully using connected-RPA to transform their businesses, solve challenges specific to their industry and improve the experience they deliver to their customers. Explore the inspiration now.

The Digital Exchange is an online “app store” for finding and downloading pre-built skills and applying them to your Digital Workforce. You can download and share AI, cognitive and advanced RPA technologies to take your automations to the next level, and with your own Enterprise DX, you can control your ecosystem while unleashing the collaboration across your teams. Explore the DX now.

Are you new to Blue Prism? We are calling all new users to learn about the ROM. 

Register to enroll on the ROM Essentials program today!