topic Re: Uploading file to Sharepoint Folder in Product Forum

Uploading file to Sharepoint Folder

john.hammond — Thu, 14 Jul 2022 09:12:00 GMT

Good Morning all,

I'm looking to upload/copy/save an Excel file to a specific Sharepoint Folder. Firstly, my apologies in advance - I am by no means an expert in working with SharePoint. I'm aware that there is the Graph API for interacting with SharePoint, but I have absolutely no idea how that works in practice (is there an idiots guide somewhere as to how this integrates with Blue Prism?). I'd like to think that there was a simpler solution to this particular issue anyway - denoting a save path on SharePoint for the document from with the 'Save Workbook As' object within the Excel VBO - but I cannot seem to get this to work.

So actually, I guess there are two questions that I currently have:
* Is there a simple way to save an Excel file to a specific folder on SharePoint? If so, how?
* How does Graph API work for someone who's not overtly familiar with SharePoint and indeed API's in general? The documentation is fine but could benefit from some example structures within Blue Prism.

Not sure if it over-complicates things, but the file(s) I'll be looking to upload will just under 6mb - from what I understand, there may be a 4mb limit on the Graph API Upload File action?

I should add: one thing I've seen mentioned is about uploading to OneDrive and then syncing to SharePoint, however unfortunately, my organisation has blocked this.

------------------------------
John Hammond
------------------------------

RE: Uploading file to Sharepoint Folder

Neel1 — Thu, 14 Jul 2022 11:58:00 GMT

Hello John - What we are doing as of now in my org is using the sync button for SharePoint file. but i see that is blocked for you. this step is making our life much easy as every automation we have this requirement to work on SharePoint.

------------------------------
Neeraj Kumar
Technical Architect
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Thu, 14 Jul 2022 15:21:00 GMT

Hi @john.hammond,

Most of the straight upload functions in Microsoft Graph have relatively small file size limitations. 4MB seems to be the default. To address this, Microsoft supports a feature known as resumable uploads or upload sessions. An upload session is basically a dedicated temporary URI endpoint for you to upload any size file to. To use an upload session there are a series of requests that you make to Graph:

Request creation of the upload session. Assuming this is successful, the new URI is returned to you.
- You'll provide the following details:
  - Site ID - The unique ID of the specific SharePoint site.
  - Drive ID - The unique ID of the SharePoint drive that will hold the file.
  - Folder Path - The path within the SharePoint drive where the file will reside.
  - File Name - The name of the file as it will exist on SharePoint
  - Auth Token
Upload the actual file using the new URI as the destination.

There's also an action available for canceling an upload session in case something has gone wrong.

There's an asset on the DX called Microsoft Graph API - SharePoint Upload Sessions that can be used for this actions. To use this connector you will need to have application registration within your Azure AD as well as a Client ID and Client Secret for the OAuth authentication and authorization. That app registration will also need the appropriate Microsoft Graph permissions applied to it. Generally this means Sites.ReadWrite.All when using Application Access credentials.

Hope this helps.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

DanielLombardo — Tue, 19 Jul 2022 12:37:00 GMT

I have been dealing with a similar situation of trying to upload larger files to SharePoint. In working with the Graph API asset I'm having difficultly getting the Drive ID. I am not a SharePoint expert and I'm not sure how to go about getting that identifier, any directions on how to get the Drive ID would be appreciated.

______________
DJ Lombardo

------------------------------
Daniel Lombardo
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Tue, 19 Jul 2022 14:01:00 GMT

Hello @Daniel Lombardo,

There are a few ways to get the Drive ID. The easiest is to use the Get Default Drive Details or Get All Drives actions on the Microsoft Graph - SharePoint connector. All you have to do is pass in the Site ID of the specific SharePoint site and it will return details of the drives associated with that site.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

sochar.k — Thu, 21 Jul 2022 10:11:00 GMT

Hello,

What about scenarios where the uploaded file are really big (for ex. 1 GB) and BluePrism returns a System.OutOfMemoryException?
I know this is more related to machine memory than to the objects themselves, but maybe someone has already solve this type of case?

Thanks

------------------------------
Kacper Sochar
------------------------------

RE: Uploading file to Sharepoint Folder

KrishnaElapavul — Thu, 21 Jul 2022 12:35:00 GMT

Hi ,
I too have same kind of scenario which runs in to several complications.
the best solution i believe is sync the Share point folder to machine where your robot runs , so you can just copy the files and can avoid all steps to access/upload to SP.

------------------------------
Krishna Elapavuluri RPA Solution Lead
TEchnology Consultant
DXC.technology
Asia/Kolkata
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Thu, 21 Jul 2022 12:44:00 GMT

Hi @KacperSochar,

Yes, this is a limitation in the current implementation of the SharePoint Upload Sessions connector. Presently, it attempts to read the entire contents of the file into memory before chopping the binary array up into chunks for sending to SharePoint. We have an item in our backlog to change this behavior, and I hope to see that update pushed to DX in the next week or so.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

PvD_SE — Fri, 22 Jul 2022 08:51:00 GMT

Hi John,

Alternative approach:
We use SharePointOnline (SPO) as well as OneDrive. If you have the same setup, then you could try this:
Once you have used a SPO link on your computer, it will probably have been mapped up within your OneDrive as well. As such, the OneDrive SPO link has a shortcut in style with:
C:\Users\Paul\OneDrive - Paul\SPO Folder\SPO Subfolder\...

Instead of having the process use SPO via a browser, it can use SPO via its shortcut. That way, you avoid interacting with an ever changing SPO interface that shows new popups every second week and requires you to login again and again without end, not to mention the infamous Sync button messing up your folders and files.

------------------------------
Happy coding!
Paul
Sweden
------------------------------

RE: Uploading file to Sharepoint Folder

john.hammond — Fri, 22 Jul 2022 09:00:00 GMT

Hah, cheers for this Paul - I literally tried this approach yesterday and had it work for the first time overnight! Thanks!

------------------------------
John Hammond
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Fri, 22 Jul 2022 21:33:00 GMT

@KacperSochar,

We've pushed an update to the SharePoint Upload Session connector on the DX that addresses the Out-of-Memory issue with large files. You can find it here. We've tested it with files up to 1.2GB without issue.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

andber — Fri, 05 Aug 2022 09:03:00 GMT

"I should add: one thing I've seen mentioned is about uploading to OneDrive and then syncing to SharePoint, however unfortunately, my organisation has blocked this."
Can you perhaps specify this part?
You are unable to open the folder in sharepoint and click the sync button, so that a local folder path is created using onedrive?
I ask since your prasing seems to suggest uploading to a onedrive folder first, then sync to the sharepoint folder.
While our method is to sync the sharepoint folder to the machine directly, using onedrive of course, but it's a one step process.

------------------------------
Leon André Bergman
------------------------------

RE: Uploading file to Sharepoint Folder

sochar.k — Thu, 11 Aug 2022 08:45:00 GMT

Hey @ewilson

Thanks for the update. I have already tested the solution, it works quite well for a 1.6 GB file.
I tried for a larger one (2.5 GB) but got this error below:

Internal : Could not execute code stage because exception thrown by code stage: The format of value ' bytes 2146959360--2147352577/2504961164' is invalid.

Do you know what is the file size limitation for upload session?
Is there any recommendation for chunks size in terms of upload session performance?

Thank you!

------------------------------
Kacper Sochar
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Thu, 11 Aug 2022 11:52:00 GMT

Hello @KacperSochar,

This is the only guidance Microsoft provides with regards to chunk size:

In our testing, we left the default chunk size of 327,680 bytes. For large files that creates a very chatty upload (meaning lots of small part which isn't ideal. For a larger file you can increase the chunk size, but you need to make sure it's a multiple of that 320KB size. You might try a multiplier of 5 or 10 and see how that works.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Wed, 17 Aug 2022 15:09:00 GMT

Hi @KacperSochar,

Have you had a chance to tinker with your chunk sizing?

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

sochar.k — Thu, 18 Aug 2022 09:04:00 GMT

Hey @ewilson,

Thanks for your reply.
Yes, I played s bit with the chunks size. This improved the upload performance a bit, but the file size limit is still there - the largest file we were able to upload was 1.6GB...

------------------------------
Kacper Sochar
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Thu, 18 Aug 2022 18:27:00 GMT

Hi @KacperSochar,

I've run some additional tests using a 10GB file and I'm seeing the same failure you are. Funny thing is the failure occurs at a remarkably similar point in the upload to what you're seeing. My tests are failing right around the 2146304000-2145386497/10737418240 chunk.

We'll research this some more to see if we can pinpoint what's triggering the failure.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

sochar.k — Mon, 29 Aug 2022 13:29:00 GMT

Hey @ewilson,

Do you have any updates related to this issue?

Thanks,

------------------------------
Kacper Sochar
------------------------------

RE: Uploading file to Sharepoint Folder

ewilson — Tue, 30 Aug 2022 20:40:00 GMT

Hello @KacperSochar,

Nothing to report at the moment. We're still investigating.

We have identified the issue, and a new release (v2.1.1) has been pushed to the Digital Exchange. We have successfully tested the upload with a file in excess of 3GB w/o issue.

Cheers,

------------------------------
Eric Wilson
Director, Integrations and Enablement
Blue Prism Digital Exchange
------------------------------

RE: Uploading file to Sharepoint Folder

saivikas729 — Thu, 01 Sep 2022 14:46:00 GMT

Hi All,

If I may add - I had similar scenario - where I had to upload the file to SharePoint.

Solution : Assuming your robot works on file and saves it locally - Make your Process send this as attachment to your mail.
Set up a Power Automate flow (free and simple) - to place this file in SharePoint whenever you receive a Mail with specified trigger and attachment.

I think this only works if file is of not very large size - But that is the case with most of the scenarios? files will not be huge all the time 🙂

------------------------------
Kolli Sai Vikas
------------------------------