Timeframe and Context

The project was carried out from April 2024 to November 2024 as part of my master’s thesis in cooperation with Soloplan GmbH.

The goal was to reduce the workload of customer support by developing an AI-supported assistant for the CarLo software product.

As the sole developer, I bore full responsibility for the project and was responsible for the design, development, and integration of the assistant into the existing software.

Implementation and Tech Stack

For the implementation, a backend based on RAG and Large Language Models was developed, which was connected to the CarLo software via a REST API. In addition, a user-friendly UI was created. A test framework was also implemented that automatically evaluates the quality of responses.

The tech stack included C# for the frontend, Python for the backend, and Microsoft Azure for hosting, LLM, database, and vector search.

Challenges and Results

The size and scope of the database posed a key challenge and required the use of chunking and vector search. It was also essential to avoid hallucinations while ensuring high response quality, for which targeted Prompting strategies, RAG, and an automated test framework were used.

The assistant was successfully integrated into the CarLo software, delivered to pilot customers, and is currently in trial operation. At the same time, it is being continuously delivered to additional customers.