Integration of Large Language Models into a Furhat robot

Timeframe and Context
The project was carried out from April 2025 to July 2025 at the Institute for Applied AI and Robotics.
The goal was to provide an intelligent Furhat robot for AI-driven dialogue applications and demonstrations at exhibitions.
As the sole developer, I bore full responsibility for the project, from backend integration to robot dialogue logic.
Implementation and Tech Stack
The implementation involved integrating a Large Language Model (LLM) into the Furhat robot. Therefore, a RAG pipeline was set up and made available via a REST API. Python and Ollama were used for this purpose.
The robot dialogue logic was implemented in Kotlin and integrated with the API. In addition, a supporting UI was developed and a test framework was created to evaluate the quality of responses in order to minimize hallucinations.
Challenges and Results
The integration of the LLM API into the robot dialogue logic was a key challenge. Prompting strategies, RAG, and evaluation tools were used to ensure the reliability of the responses.
The integration was successfully implemented: The robot was demonstrated at the Automatica exhibition and now serves as a research platform for AI-supported dialogue systems.