Blue Prism is a Windows OS application built using .NET, it can only be installed on the windows operating system.
To interface with a Mainframe you need to interface with a Hllapi complient Terminal Emulator (such as IBM PComm, Rumba, Attachmate, Powerterm, etc) application running in Windows but connecting to the Terminal Server. There is a Mainframe Automation guide and lots of knowledgebase articles about how to interface with a Mainframe.