Generative AI in SAP On-Premise: Working with SAP data in the local chatbot

 

Where does generative AI bring real added value in connection with business data from SAP? The first areas of application can be seen in chatting with SAP data. For example, business data such as article masters, documentation, parts lists or project structures can be queried. A second scenario can be seen in the easier operation of complex SAP processes: Natural language can be used to call up, change or even create sales orders, projects or products. An internal decision must be made in advance as to whether the chatbot can access company-specific data and thus extract and process knowledge from it. At this point at the latest, the question arises: cloud or on-premise?

On-premise or cloud: security versus innovation?

Many companies use SAP on-premise because the security-related advantages outweigh the disadvantages for them: despite higher resource requirements, they offer greater control over sensitive data. At the same time, local systems can be tailored to the individual needs of the company with a corresponding amount of effort, which is another argument against using cloud systems. The following also applies when using AI: the further the data is located in the cloud, the less control over data security remains with the company. The fact that AI developments from Walldorf focus almost exclusively on cloud tools is a hot topic of discussion in SAP circles. When it comes to AI, companies that use SAP on-premise are faced with the options of switching to the cloud, introducing a different ERP system or remaining cut off from AI functions.

Server farms mostly unnecessary

With those companies in mind, the Milliarum AI Construction Kit was recently launched on the market. This enables users to add AI-based functions to their SAP system. With preconfigured functions for OpenAI, Azure OpenAI and locally operated open source models, chatbots can be rolled out quickly in companies. The first application-specific chatbots for SAP On-Premise are set to follow shortly. Companies will be able to select their large language model and connect it locally to their own AI system. After all, you don't need a gigantic chatbot that speaks over 100 languages and has been trained with several hundred billion parameters to chat with your own SAP data if only German is ultimately to be used for limited use cases. A server farm should therefore not be necessary for most use cases in the SAP environment.

Open source LLM available

Once the objectives and application scenarios have been defined, the use of AI in the SAP environment requires the selection of a host that meets the company's security requirements. On the one hand, you can opt for the big players on the market: OpenAI is an American cloud-based hoster, Microsoft Azure offers comparable functions and fulfills European requirements. Local open source large language models (LLM) on the company's own hardware are an alternative. These can be found for various use cases on the Hugging Face platform, for example. The Fraunhofer Institute recently recommended German-language optimized models based on Mistral 7b. It already runs on hardware that is similar to a gaming PC in terms of performance requirements and is available from around 10,000 euros. The open source model Mixtral 8x22b is currently the most powerful European LLM, fluent in five languages and trained with over 140 billion parameters. It requires at least 300 GB of graphics memory for productive inference operation, which does not necessarily have to be provided by an Nvidia data center. Even older generations of NVIDIA graphics cards are perfectly adequate in terms of performance and can be used cost-effectively.

Ecological footprint is rather small

In addition to the monetary aspect, choosing an open source model on your own hardware saves a lot of electricity compared to the huge data centers that usually underlie cloud-based applications. They do not require the company's own hardware and are usually based on a usage fee depending on the number of words entered. Depending on the application scenario, companies quickly reach a significant cost factor. Cloud-based systems can be up and running after just one day with the right preparation. Local applications, on the other hand, require the hardware to be purchased. It must then be integrated into the company network and the AI model must be set up and trained with the company's own parameters depending on the intended use. Although the introduction is complex, the operation meets the company's own security requirements and the running costs are significantly lower than with a cloud-based AI system, where every API access is remunerated.

Objective autonomous SAP processes

The better the basis of the SAP data, the more powerful a generative AI based on it will be. Preparing the input data is a step on the way to AI commissioning that should not be neglected. However, SAP data forms a comparatively simple basis for such systems. After all, the business data is already available in the SAP software. However, it must first be extracted from the system in order to process it and feed it back into the software. Experts are currently also concerned about the extent to which the use of AI will actually bring productivity benefits in the SAP environment at the end of the day. Bottleneck resources in particular are showing rapid optimization potential. While AI is currently mainly used as a pure chat function and in programming environments for creating evaluations directly from SAP data and for creating and changing SAP objects, it should be possible in future to operate complete processes in SAP based on natural language. Data is stored in such a way that the AI can access this data via training. It is already foreseeable that processes, including machines, will increasingly be controlled automatically in the background by AI and only the chat will represent the application by the user. Nevertheless, the use of generative AI in the SAP environment is still in its infancy: the actual benefits, whether cloud-based or on-premise, have yet to be proven.