AI Engineering Lab

Most AI portfolios show a demo. A demo tells you a system can work once. It does not tell you how often it works, or what happens when it should say “I don’t know”.

This lab is my answer: three small systems, each built one concept at a time and each shipped with an eval harness, so every claim in it has a number behind it.

• A knowledge assistant (RAG). Ingest documents, retrieve with semantic search plus keyword search merged by Reciprocal Rank Fusion, rerank with a cross-encoder, and answer with citations or refuse. It has a streaming web UI.
• An agent platform. An agent built from its atoms (a model, tools, a loop) and then on LangGraph.js, with human approval for risky actions, guardrails, per-node cost and latency traces, and checkpoints.
• An MCP assistant. An MCP server exposing codebase tools, and a client that bridges them into a function-calling loop for codebase Q&A with file and line citations.

A platform layer then composes them: the agent consults the RAG system through a tool.

• Retrieval first. Hybrid retrieval, then a cross-encoder reranker, behind a small vector-store interface. An in-memory store came first and a pgvector store followed, each with its ADR.
• Refuse when ungrounded. A relevance floor and an “I don’t know” path, tested with questions the documents do not cover.
• Prompt-injection defence. Documents are treated as data, and the harness includes injection probes.
• Measured, with a judge. Retrieval recall is deterministic. Answer quality uses an LLM judge, cross-checked by a judge from a different model family, and the agent harness checks that a risky tool pauses for approval.
• Provider-agnostic. The model sits behind a factory over an OpenAI-compatible API, so the stack runs on a free tier.

Each project is a ladder of runnable rungs, one concept per rung, with an ADR for each real decision. CI runs unit tests and typecheck for all three flagships, plus a deterministic eval-regression gate for RAG recall.

• The headline number did not hold. The jump from 93% to 100% recall@1 came from a 15-question set. When I added a second domain and grew the set to 30 questions, recall@1 was 97% for all three retrievers, and the reranker’s benefit moved to recall@3. I kept both results in the write-up.
• Small, authored corpora. Both domains are written by me, not scraped, so the results are directional rather than a benchmark. The judged metrics on the larger set were still pending a token-budget refresh.
• pgvector is not the default. The store is shipped and tested, but the in-memory store is still the default backend (ADR 0006).