A GraphRAG system on Neo4j, benchmarked against a strong hybrid retrieval baseline on 2WikiMultiHopQA. The goal is a fair answer to one question: when does a knowledge graph actually help multi-hop question answering, and when does it not? The hypothesis was pre-registered and the held-out test split was run once: GraphRAG answered 0.79 of 375 test questions exactly, the hybrid baseline 0.57.
Exact match 0.57 (hybrid RAG) to 0.79 (GraphRAG) on 375 held-out test questions: +0.22 (95% CI +0.17 to +0.27) · hypothesis pre-registered, test split run once · total LLM cost 1.82 USD · 12,547 relations from 2,049 documents · extraction recall 0.75, slot precision 0.89 (strict, full run, 247 dev triples)
Python
Neo4j
Knowledge graph
E5
BM25
Cross-encoder reranker
gpt-oss-120b
Docker
How it works
2WikiMultiHopQA: 2,049 paragraphs, 500 questions in dev and test splits
LLM entity and relation extraction with strict JSON schema output: 12,547 relations for 0.35 USD
Entity resolution keeps same-name entities apart and records why every merge happened
Hybrid baseline: BM25 + E5 dense + bge reranker, the retriever from the MSc thesis
Short answers from gpt-oss-120b, pinned provider, every LLM call cached and logged with cost
EM and F1 with 95% bootstrap confidence intervals, quality bars fixed before results
Test split run once with frozen settings: paired EM difference +0.22 (95% CI +0.17 to +0.27), pre-registered expectation met
A service that reads freight emails in German and English, extracts shipment details with an LLM and either auto-accepts them or routes them to a human review queue. The model extracts, plain rules decide, so the model never makes the business decision.
175 tests · retries with backoff · prompt injection defence · dangerous goods always go to a human
Python
FastAPI
SQLite (WAL)
Gemini
Pydantic
pytest
How it works
POST /emails, idempotent deduplication by Message-ID and content hash
Job queue in SQLite, worker claims one due job atomically
Gemini extracts fields with per-field confidence, email wrapped in random delimiters
Deterministic validation rules decide: auto-accept or human review
Retries with backoff, attempts history, stuck-job recovery
Review queue to approve, correct or retry, plus a German and English eval set
BM25 sparse search fused with Microsoft E5-base-v2 dense embeddings across 8.84M MS MARCO passages. The fusion weight (alpha=0.70) was discovered through 11 experiments (results in RESULTS.md) and validated with a paired t-test, not assumed.
93% Recall@10 · +2.9 pts vs alpha 0.5 (p = 0.002) · +11.4 pts vs BM25 only
Live map of all roadworks across Germany. Parallel async pipeline fetches 3,197 live sites from the official German government Autobahn API in under 5 seconds. Groq LLM generates plain-English impact summaries per site automatically. Built and deployed in 6 days.
3,197 live roadworks · 5 second load · LLM summaries · Railway + Vercel
Custom n8n node in TypeScript that brings live German Autobahn data (roadworks, warnings, closures, EV charging, lorry parking, webcams) into any workflow. It queries many roads in one call and turns a messy API (booleans as strings, text arrays, a different key per event type) into clean, typed items. Works as a tool for the n8n AI Agent, and the compact output saves tokens.
612 live roadworks in one request · 10 unit tests · official n8n linter passing · CI on every push
Workflow running on n8n Cloud. A webhook receives a ticket, Gemini extracts category, priority, sentiment, a summary and a draft reply as structured fields, and a Switch node routes it to Slack, email, my own RAG API or a queue. The model classifies, plain logic routes. Retries handle model overload, and if the knowledge base is down the reply falls back to the AI draft.
4-bit quantisation with PEFT LoRA adapters on a free Colab T4 GPU. Trained 0.089% of parameters. Evaluated with RAGAS using Gemini-as-judge. Published on HuggingFace Hub.
Loss 2.47 → 0.89 · 64% reduction · 0.089% of parameters trained
Four specialised LangGraph agents: Search (DuckDuckGo), Summarise (temp 0.3), Fact-Check (temp 0.1), Writer (temp 0.4), each tuned for one job in the pipeline with stateful graph orchestration.
End-to-end RAGAS evaluation pipeline with Gemini-as-judge tracking Faithfulness, Answer Relevance, and Context Precision across ten NLP test questions. Live radar chart on HuggingFace Spaces.
Faithfulness 0.909 on 10 test questions · Answer Relevance 0.882 · overall RAGAS 0.877
For 20 dev questions of 2WikiMultiHopQA (5 of each multi-hop type), the knowledge-graph paths the GraphRAG system used as facts. Pick a question: the view zooms to its path and dims the rest. Hover or tap a node for its name.
personfilmworkplaceorg
The graph loads when this part of the page comes into view.
Answers from the committed dev run; "correct" is an exact match, other wordings were checked by the evaluation judge. Data and the script that builds it: graphrag-engine. Names and relations derived from Wikipedia and Wikidata via 2WikiMultiHopQA.
MSC THESIS FINDING
The fusion weight
Sparse and dense retrieval are not equal partners. Assuming they are costs you accuracy.
Most hybrid RAG implementations split BM25 and dense embedding retrieval 50/50 by default. I ran eleven systematic experiments and found the best tested value at alpha=0.70, confirmed with a paired t-test (p=0.002). That calibration choice is worth +2.9 pts over alpha 0.5 (p = 0.002, 95% CI +1.1 to +4.7), +11.4 pts over BM25 only, on 8.84M passages.
The common assumption of alpha=0.50 was tested directly and found to be suboptimal.
InstitutionUniversity of Europe for Applied Sciences, Potsdam
Corpus sizeMS MARCO 8.84M passages
Queries100 stratified MS MARCO dev queries
Dense modelMicrosoft E5-base-v2
Sparse methodBM25
Experiments11 experiments (results in RESULTS.md)
VariableFusion weight alpha (0.0 to 1.0)
Statistical testPaired t-test, t(99) = 3.14, p = 0.002
vs alpha 0.5+2.9 pts (95% CI +1.1 to +4.7)
Best tested alpha0.70 gives 93% Recall@10
Replay the experiment
Move alpha and see the measured Recall@10. 0 means BM25 only, 1 means E5 only.
alpha 0.5Hybrid
90.1%
Recall@10
vs BM25 only+10.4%
The common default. Try 0.7.
Source: RESULTS.md, alpha grid search. 100 stratified MS MARCO dev queries. Alpha 0.1 and 0.2 are left out because two result files in the repo disagree on them. "vs BM25 only" is the relative change reported in RESULTS.md.
All numbers as a table
ALPHA
METHOD
RECALL@10
VS BM25 ONLY
0.0
BM25 only
81.6%
baseline
0.3
Hybrid
87.2%
+6.8%
0.4
Hybrid
88.9%
+8.9%
0.5
Hybrid
90.1%
+10.4%
0.6
Hybrid
91.8%
+12.5%
0.7
Hybrid
93.0%
+13.9%
0.8
Hybrid
91.2%
+11.7%
0.9
Hybrid
89.5%
+9.7%
1.0
E5 only
82.5%
+1.1%
ENGINEERING DIARY
Decisions and failures, with receipts
Real entries from the repos, newest first. Each one links to the commit or document it comes from.
GraphRAG EngineRESULT
The test split, run once: pre-registered expectation met
One primary result was fixed before the run: the paired exact-match difference, GraphRAG minus the hybrid baseline, over all 375 test questions, expected to have a 95% CI above 0. The settings were frozen and tagged, and a lock lets the test script run only once. Result: 0.57 for hybrid, 0.79 for GraphRAG, a difference of +0.22 (95% CI +0.17 to +0.27). The expectation was met.
Three criteria were fixed in the plan before any result, and all three passed on the 125 dev questions. Multi-hop questions with all gold paragraphs in context: +0.15 over hybrid (bar: at least +0.10). Paired exact-match difference: 0.18 (95% CI 0.10 to 0.26; bar: CI low at least -0.05). Exact match on the 50 bridge-comparison and compositional questions: 0.60 against 0.12 for hybrid.
Entity resolution re-run passes: 49 of 50 merges correct
After the failed run 1, the re-run with judge prompt v2 and the cannot-link rule was checked on 50 sampled merges: 49 were correct, against a bar of 48 fixed beforehand. The first pass of the labels was drafted by a second model, then reviewed and corrected by me.
Judge prompt v2, a cannot-link rule and a pilot in both directions
Each side of a candidate pair now states its role ("the subject of its paragraph" or "mentioned in a paragraph about X"), and the judge decides on the named entity only. Union-find refuses a merge when any pair across the two clusters was judged "different", and merges run in a fixed order. The new pilot tests both directions: at least 22 of 24 known bridge links must be judged "same", and the bad merges from run 1 must be judged "different". It passed both gates. The full re-run is in progress.
Entity resolution run 1 failed, and it stays on record
Run 1 judged all 17,309 candidate pairs for 0.59 USD. It merged the United States with the United Kingdom (117 mentions), and bridge link recall on the report split fell from 0.83 to 0.51. Two causes: the judge prompt confused a mention with the subject of its paragraph, and there was no cannot-link check, so 2,439 pairs judged "different" still ended up in one cluster. All outputs are kept in results/m3/run1_flawed.
Before writing entity-resolution code, the plan and its quality bar went into docs/PLAN.md: at least 48 of 50 seeded-random merges correct on the report split. After the first pilot, auto-merge was turned off: dev gold triples give no same-name negatives, so the merge threshold cannot be tuned from gold.
Prompt v3 failed one bar, and a rule fixed in advance decided
The extraction quality bar (recall of at least 0.75, slot precision of at least 0.90, at least 27 of 30 hand-checked triples correct) and the rule for a failed prompt v3 were both committed before the run. On the pilot's 9 fresh batches (21 dev triples), prompt v3 passed recall (0.76) and failed slot precision (0.76). All 10 failures were classified by hand: 1 real miss, 0 wrong facts. The hand check found 28 of 30 triples correct. Under the rule, the full corpus was extracted with prompt v3 and the strict score is reported as it is.
Re-queue a failed email without rewriting its history
After a quota or API outage, POST /review/{id}/retry re-queues a failed email. The attempts counter is not reset, because it is the lock token and the attempt number in the history; retry_base records where the new budget starts. After a requeue with MAX_ATTEMPTS=8, the log shows attempt=9/16.
Two independent checks send dangerous goods to review: the model's cargo class, and a plain-code marker check (ADR, Gefahrgut, UN numbers, hazmat, IMDG). Negated mentions such as "kein Gefahrgut" do not trigger it, and ADR is matched in capitals only, so "Adr." (Adresse) does not either. The check errs toward review: a false positive costs one review, a false negative auto-accepts dangerous goods.
The model only fills in fields and a confidence per field; the output schema has no decision field. Deterministic code accepts an email or sends it to review with stable reason codes such as missing:weight. The model's self-reported confidence is not well calibrated, so it is one check among several.
I build, evaluate and deploy practical ML systems, from data preparation and experimentation to APIs, cloud deployment and monitoring.
01 · UNDERSTAND
Define the problem
Translate the product or business problem into a concrete ML problem with measurable success criteria.
02 · EVALUATE
Choose metrics first
Define Recall@10 and RAGAS Faithfulness before optimising. Metrics chosen for the problem, not retrofitted.
03 · BASELINE
Establish a baseline
Understand whether the new approach improves anything. alpha=0.70 gave +2.9 pts over the default alpha 0.5 (p = 0.002).
04 · DATA
Prepare and validate
Check data quality, distribution, edge cases, and leakage before training. Real-world document formats handled properly.
05 · EXPERIMENT
Track systematically
Every experiment recorded with its settings and results; 11 alpha experiments, results in RESULTS.md, validated with a paired t-test.
06 · TRADE-OFFS
Analyze trade-offs
Accuracy vs latency. The extra cost of running two retrievers must be justified by the Recall@10 gain.
07 · DEPLOY
Package reproducibly
Docker, Kubernetes, systemd. Services that restart on crash, roll out with zero downtime, behave identically in dev and prod.
08 · TEST AND MONITOR
Test and monitor
12 pytest regression tests. Prometheus /metrics and /health. GitHub Actions CI/CD that fails the build on regression.
09 · ITERATE
Improve from evidence
Use observed failures and evaluation results to drive the next experiment. Evidence from production, not intuition.
TECHNICAL SKILLS
By category
MACHINE LEARNING
Python 3.11
PyTorch
HuggingFace
Model evaluation
Experimentation
Error analysis
NLP AND LLM
RAG
GraphRAG
Knowledge graphs
Hybrid retrieval
BM25
Dense embeddings
E5-base-v2
FAISS
Rerankers
LLM APIs
QLoRA
PEFT
LangChain
LangGraph
RAGAS
Prompt engineering
DATA
SQL
PostgreSQL
pgvector
Neo4j
SQLite
Redis
SQLAlchemy
Document ingestion
Data validation
BACKEND ENGINEERING
FastAPI
asyncio
REST APIs
pytest
Pydantic
JWT auth
SSE streaming
Webhooks
TypeScript
Node.js
React
MLOPS AND CLOUD
AWS EC2
Docker
Kubernetes
GitHub Actions
Prometheus
CI/CD
systemd
TOOLS AND PLATFORMS
n8n
Workflow automation
Git
Linux/CLI
HuggingFace Spaces
Railway
Vercel
Gemini
Groq
BACKGROUND
Experience and education
TARGET ROLEAI Engineer, LLM Engineer, NLP Engineer, ML Engineer
LOCATIONBerlin, Germany · Open to relocate across Germany
WORK AUTHJob Seeker Visa · No employer sponsorship needed
AVAILABILITYAvailable immediately
LANGUAGESGerman B1 · English C1
PROGRAMMINGPython · SQL · TypeScript
April 2026 to Present
AI Engineer
Independent Projects · Berlin, Germany
Building and shipping production-style AI systems end to end, with a focus on retrieval, agents, reliable LLM pipelines and honest evaluation.
Built a GraphRAG engine on Neo4j with LLM extraction and entity resolution; on 375 held-out test questions it raised exact match from 0.57 (hybrid RAG) to 0.79, +0.22 (95% CI +0.17 to +0.27), with the hypothesis pre-registered and the test split run once
Built an LLM email intake service with a job queue, retries, guardrails, a human review queue and 175 tests
Shipped a multi-tenant AI SaaS in 7 days with auth, streaming, CI tests and monitoring
Released an open-source n8n community node in TypeScript (10 unit tests, CI) that gives AI agents clean, typed data, and an AI support ticket triage workflow on n8n Cloud
Mar 2024 to Mar 2026
MSc Data Science
University of Europe for Applied Sciences · Potsdam, Germany
Thesis: AI-Powered Retrieval-Augmented Assistant for Evidence-Based Question Answering. Hybrid RAG combining BM25 sparse retrieval with Microsoft E5-base-v2 dense embeddings. 11 experiments (results in RESULTS.md). 93% Recall@10 on 8.84M MS MARCO passages. Best tested fusion weight alpha=0.70, validated with a paired t-test, p=0.002. QLoRA fine-tuning and deployment on AWS EC2 and Kubernetes.
June 2019 to August 2023
B.Tech Computer Science and Engineering
GITAM University (Deemed) · Visakhapatnam, Andhra Pradesh, India
Key modules: Machine Learning, Artificial Intelligence, Data Structures, Database Management Systems, Web Application Development, Computer Networks, Operating Systems.
AVAILABLE NOW · OPEN TO WORK ACROSS GERMANY
Let's build something worth measuring.
Seeking AI Engineer, LLM Engineer, NLP Engineer and ML Engineer roles across Germany. Job Seeker Visa holder, no employer sponsorship needed. Available immediately, open to relocation.
