topic Re: Using Blue Prism VBO for Apache Kafka, how do we send a header value? in Digital Exchange

Using Blue Prism VBO for Apache Kafka, how do we send a header value?

TaylorHuisman — Thu, 27 Jan 2022 15:03:29 GMT

In the consume and produce action, there is an input field for key and value but nothing for sending headers. We would like headers added to our message. Is there any way to access this functionality?

Re: Using Blue Prism VBO for Apache Kafka, how do we send a header value?

ewilson — Thu, 27 Jan 2022 18:58:27 GMT

Hi @TaylorHuisman,

It is possible, but it will take some rework of the code. Within the Confluent .NET library there's the Message class which is what we're creating and passing in the Produce() and ProduceAsync() actions. The Message class exposes a property called Headers that you would need to set with your specific header information. Unfortunately, due to the way the code is currently structured in the Code stages you can't just pass it in as the Message object is being created within the library call to Produce() and ProduceAsync().

So what would need to be done is to pull the instantiation of the Message object out of the function call and then just pass the actual instance of Message object to the function. Hopefully that makes sense. It would basically look like this:

This is a bit of a contrived example because of the hardcoding and limit of 3 headers, but hopefully you get the idea. I'll add an item to our backlog to get the Kafka VBOs updated to support the use of Message Headers.

Cheers,
Eric

Re: Using Blue Prism VBO for Apache Kafka, how do we send a header value?

TaylorHuisman — Thu, 27 Jan 2022 19:47:35 GMT

@ewilson thanks for the reply.

This is what we ended up doing to solve:

We were able to alter the produce action to accept a headers parameter, and include in the message contents.

The headers parameter is a string of key/value pairs that get converted in the altered produce action into header type objects. The input string is configured like such:

"Header 1:This is Header 1,Header 2:This is header 2"

We also made the following imports to the end of the global code options in the produce action:

Re: Using Blue Prism VBO for Apache Kafka, how do we send a header value?

ewilson — Thu, 27 Jan 2022 20:19:22 GMT

@TaylorHuisman,

Thank you for the feedback. I also noticed your review of the Avro connector on the DX. At the time these assets were developed there was an issue that required us to roll back to an earlier version of the Confluent .NET library which in turn necessitated the use of the older System.Memory DLL. We've since built newer versions of the VBOs on the current 1.8.2 stable release of the Confluent library which uses a newer version of System.Memory.

We're going back now and and adding support for Message Headers, but we'll be going the Collection route for passing them in. Happy to discuss this decision if you're interested. Hopefully we'll have the new assets tested and posted to the DX sometime next week.

Cheers,
Eric

Re: Using Blue Prism VBO for Apache Kafka, how do we send a header value?

ewilson — Sat, 29 Jan 2022 04:40:50 GMT

Hi @TaylorHuisman,

FYI - We posted v1.1.0 of the Apache Kafka w/ Avro VBOs today. The new VBOs include support for message headers. We also reworked the support file archives. Now there are essentially 3 zip files:

Core Kafka libraries (apply to any version of BP from 6.5 to 7.0)
Support libraries for BP versions 6.5 to 6.7
Support libraries for BP versions 6.8 to 6.9
If you're running v6.10 to v7.0, you only need to core Kafka libraries.

Cheers,
Eric