SAP Monitoring with ChatGPT? (July 2023)
This article is authored by Brian Bates, Director of Solutions Architecture at RHONDOS.
In my last article, I walked us through how to develop your own custom command in #Splunk to pass prompts directly to the #OpenAI API using any of their LLMs. This custom command is pretty cool because you can apply the function to user-entered prompts like “what is the best SPL to search for #SAP IDOC failures?” or tokenize the results of search and automatically pass them through to the API.
We know the use cases for #ChatGPT are endless, so in this article, we’ll address the management of these integrations as organizations pilot, prototype, and move more and more LLMs into production workflows.
At #SAP #Sapphire last week, our team attended the most important sessions related to customers’ problems of today and those that are yet to be identified tomorrow. SAP announced the launch of AI solutions for “future-proofing” supply chains, and a partnership with Microsoft to achieve this goal. While we fully realize AI will not be replacing existing supply chains anytime soon, the number of integrations between existing business flows and LLMs like OpenAI’s GPT-4 are growing at a break-neck pace.
We wanted to rapidly prototype what it could look like to monitor these new integrations as they begin in your environment. For our demonstration, a trial version of SAP BTP was spun up. We used the intuitive interface to write an iFlow that passes prompts to OpenAI and receives the responses on the SAP Cloud Foundry platform. There are many ways to configure these integrations and we anticipate customers to implement GPT models at various stages in the business process, most of which won’t require human authored prompts.
Using SAP PowerConnect Cloud, an observability pipeline with many additional ETL features optimized for real-time data processing, the API is monitored for key performance trends. This monitoring solution operates externally to SAP BTP and enables visibility for the platform holistically. In the most recent version, SAP PowerConnect Cloud can even be hosted on a Splunk Heavy Forwarder, reducing the infrastructure requirements.
Gathering all of the important metrics and log files in Splunk makes monitoring a ChatGPT integration very straightforward. When it comes to performance, the technology owner for OpenAI needs to be acutely aware of the token being used for consumption monitoring and maintain SLAs to the user base. Additionally, we can monitor the temperature of the prompt conditions. Temperature in the OpenAI API is a gradient for how explicit or creative you are expecting the return responses to be.
SAP ChatGPT Monitoring in Splunk with RHONDOS & PowerConnect
Here we’ve built a centralize monitoring tool using Splunk Dashboard Studio to visualize and report on key information about this ChatGPT integration using SAP’s Cloud platform – all the way down to the use base: who is using the dashboard the most, and will this solution be adopted across functional areas of the organization.
RHONDOS is the North American authority in splunking SAP. We were the first to publish reference code for custom Splunk commands integrating OpenAI and continue to lead the charge in the latest innovations from SAP and Splunk. If your company is interested in integrating ChatGPT into Splunk or the SAP Cloud Platform, let us know.