Using LLMs as Blue Prism development sidekicks - what's your experience so far?

[Michael_S]

Hello friends!

I'm currently developing a few community automations with BPE and, in the spirit of vibe coding, decided to use Google's AI Studio as a sidekick. I know I'm not the only one, many of you I've spoken to over the past few months have adopted an LLM as a development "buddy".

It's both saved me time, and caused me frustration, and I wanted to start a discussion here to see if that's a shared experience.

So - what's your experience been like, using LLMs whilst creating automations in Blue Prism?

You might like to share:

• What kinds of tasks are you asking LLMs to complete during the development process?
• Do you have any specific examples of how an LLM saved you time, or helped you overcome a blocker?
• Or have you faced issues instead? Hallucinations, incorrect information, poor quality work?
• One area I'm particularly interested in - how knowledgeable do the models seem to be about Blue Prism products? Is their understanding up to date?

Really looking forward to hearing from you all - and I'll share some of my experiences later on in the chat too 💙

[Tejaskumar_Darji]

From my experience so far, the current major LLMs don’t fully understand the Blue Prism expression builder and its syntax, but they often suggest good expression ideas; however the generated expressions don't directly work in BP as it will throw syntax errors but with basic BP knowledge you can easily fix syntax errors and use the LLM generated expression with BP to save time. However, the responses are well aligned with your natural prompt and can serve as a helpful starting point.

On the other hand, LLMs perform quite well with .NET and C# code stages. You can directly use the code they generate and integrate it into your Blue Prism code stages. When prompted, they also assist with defining input and output parameters, which we commonly use in BP code stages to maintain flexibility.

Below are 2 pointer area I haven’t explored yet, but seems worth trying -

1) Generating documents like PDDs, SDDs, High Level Flow Diagram etc directly from the Blue Prism code. If anyone has attempted this, I’d be interested to hear how it worked out.

2) Another potential use case is leveraging LLMs to understand or interpret Blue Prism code developed by other developers. This could help in quickly understanding the logic, identifying issues, or even suggesting improvements. E.g. if a developer has missed a critical exception handling scenario, the LLM could potentially flag it, making it a valuable help in code reviews and quality checks.

[Alex762]

We're a small team, just getting started with bp, so we have been using AI to help us learn the platform and develop our first projects. We found that for basic problems, gpts do fine in helping us resolve them.

But if it's something complex, we'll often get an answer that is based on outdated information (telling us about buttons and menu items that dont exist for example).

We are still learning how to effectively prompt which may be part of the problem, though. If anyone has any tips on that, it would be very useful!

[david.l.morris]

I have made some small attempts to use Blue Prism process/object XML as reference for an LLM to generate valid Blue Prism code (XML), but I haven't had success on that yet. Of course, I do not spend much time playing with AI much. I know I'll need to eventually, but the only reliable use of LLMs that I've come across so far is using something like GitHub Copilot or Microsoft Copilot with them having the full context of the code being worked with. Something in the same vein seems like it'd be great for Blue Prism since a lot of the time comes down to just busy work.

A real problem though is that I don't know how I'd train the LLM on the code base including custom VBOs and their actions without the LLM being able to directly query the BPAProcess table and also be able to interpret it properly.

I imagine some people could have limited success using a chatbot to get suggestions, but that kind of decontextualized interaction with chatbots is far more likely to produce hallucinations.