My portfolio had a card that said "RAG Document Assistant (Coming Soon)". Under it was a tech stack: LangChain, Pinecone, Chroma, GPT-4, Claude. Under that, a results section: 90% retrieval accuracy, positive feedback from beta testers, targeting a Q2 2025 release.
This week I ran a grep on my own repository and found app/api/rag/query/route.ts. Ninety-six lines. A working API route that takes a question, retrieves the three most relevant passages from a set of FDA medical device guidance documents, and answers from them with a prompt that refuses to guess when the passages don't cover the question.
It had been sitting there for months. It was not coming soon. It was done, and it was better than the card describing it.
What RAG is, in plain words
Retrieval-Augmented Generation is a two-step way to make a language model answer from documents it was never trained on. First you retrieve: cut the documents into chunks, turn each chunk into an embedding (a vector of numbers that captures its meaning), and store those vectors somewhere you can search. When a question arrives, you embed it the same way and pull the chunks whose vectors sit closest. Second you generate: paste those chunks into the prompt and tell the model to answer from them.
The analogy I use is an open-book exam. A model on its own is answering from memory, and when memory runs out it invents. RAG hands it the book open to the right page, so it can cite the page and admit when the page doesn't say.
This is the technique most companies actually want from AI right now. Nobody needs a chatbot that knows the capital of France. They need one that answers questions about their contracts, their policies, their product docs, and doesn't make things up while doing it.
What the code actually did
The route I found was specific in every place the portfolio card was vague.
The domain was FDA compliance guidance for medical devices, held in a knowledge base file. The text was split into 500-character chunks with 50 characters of overlap, so a sentence cut at a chunk boundary still appears whole in one of its neighbours. Each chunk was embedded with OpenAI's text-embedding-3-small, the cheap one, and stored in HNSWLib, an in-process vector index that lives inside the Node server. Queries pulled the top three chunks and handed them to gpt-4o-mini with a system prompt that said: answer only from this context, cite the section, and if the context doesn't contain enough to answer, say so.
Every one of those is a decision. The chunk size trades context against precision. The overlap costs storage to avoid losing meaning at the edges. The small embedding model and the mini chat model are a cost choice I could defend. The refusal instruction is the whole point: it's what separates a grounded assistant from a confident liar, and it only works because retrieval happened first.
None of that was on the card. The card listed Pinecone and Chroma, which the code never touched, and Claude, which the code never called. If a hiring manager had grepped the repo against the stack list, they would have found a mismatch on the first try.
The numbers were worse than the mismatch
"90% accuracy in document retrieval" sounds like a measurement. I did not have one. There was no test set, no retrieval metric, no evaluation harness. It was a line written to sound like a result.
"Positive feedback from beta testers." There were no beta testers.
I'm writing this down because I think it's a common failure and an invisible one. When you fill out a portfolio template, the template has a Results section, and the section wants a number, and you write something that fits the shape. Months later the number is still there, looking like evidence. The problem is not that it's a lie exactly. The problem is that the first question in any interview about this project would have been "how did you measure that," and I had no answer.
What I changed
The card now describes what exists. Not "RAG Document Assistant" but a compliance assistant that answers questions over FDA guidance, cites the section, and refuses to answer outside its sources. The stack is the real stack. The results section is empty, because there are no results yet.
That empty section is now a plan rather than a placeholder. The next version moves the index out of memory and into Postgres with pgvector, so it survives restarts and scales past a demo. It gets a test set of questions with known answers, a recall@10 number (how often the right chunk appears in the top ten retrieved), and RAGAS scores for faithfulness and answer relevance. Those go on a public evals page with the retrieved chunks and similarity scores visible per query.
When that page exists, the results section gets a number again. This time it will be one I can defend, with the harness that produced it sitting next to it in the repo.
The lesson
Two things, and they point in opposite directions.
I had undersold the work. A working, grounded, cost-aware RAG route over a regulated domain is a real thing, and I had labelled it "coming soon" because the template made me feel it wasn't finished until it matched some generic ideal.
And I had oversold the results. Numbers with no measurement behind them are worse than no numbers, because they look like the thing that's missing.
The fix for both is the same habit: describe the code that exists, and let the evals page earn the numbers. If you have a portfolio, open the repo next to it and read them side by side. Mine disagreed in five places. I'd rather find that than have someone else find it for me.