LlamaTalks
Summary
LlamaTalks is an open-source Spring Boot chatbot that keeps the language model local through Ollama while adding document-grounded answers with Retrieval-Augmented Generation. A user can ingest documents, let Apache Tika extract and split their text, generate embeddings and store them for semantic retrieval, then ask questions in a conversation that keeps persistent history. LangChain4j connects the retrieved context and conversation memory to Ollama. Reactor/Flux and Server-Sent Events stream the answer to the client instead of waiting for one completed response. The project exposes a RESTful API and supports conversation management, document listing and flexible chat or embedding model selection. The repository records 21 public commits. The README documents the architecture and setup, but it also warns that not every listed document type was tested and that changing embedding dimensions requires clearing the store and ingesting documents again.
Verified facts
- The public repository records 21 commits.
- The README identifies Spring Boot, LangChain4j, Ollama, SSE, persistent history and RAG document ingestion.
- Apache Tika parses documents before chunking, embeddings and semantic retrieval.
Technical Architecture
The request path follows two connected flows:
- Core Backend: Built with Spring Boot 3, utilizing JPA/Hibernate for structured data persistence (conversations, messages).
- AI Orchestration: LangChain4j manages the interaction with LLMs, memory management, and the RAG pipeline.
- Data Ingestion: Apache Tika is used to parse and extract text from various document formats for the vector store.
- Vector Store: Embeddings are generated locally and stored in a vector database to enable semantic search capabilities.
- 01 Document Tika → chunks
- 02 Vector store embeddings + retrieval
- 03 Ollama context + history
- 04 SSE Reactor/Flux
LangChain4j orchestration · PostgreSQL history
Model boundary
Embedding dimensions are part of the storage contract. Switching the embedding model can change that dimension, so the database must be cleared and every document ingested again before semantic retrieval is used.
Key Features
- Context-Aware RAG: Dynamically retrieves relevant information from uploaded documents to answer user queries accurately.
- Streaming API: Utilizing Spring WebFlux principles for non-blocking, real-time token streaming.
- Flexible Model Support: Easily switch between different open-source models (Llama 3, Mistral, Gemma) hosted via Ollama.
- Persistent Chat History: Full conversation history is stored in PostgreSQL, allowing context retention across sessions.
How this was verified: Read the public repository ↗