Run the steered search from Part 8:

./embed search "how does the app handle retries" --suppress "if err != nil" --top 3

1. db/retry.go      :: withRetry        sim: 0.7923
2. auth/token.go    :: RefreshToken     sim: 0.7711
3. cache/store.go   :: SetWithExpiry    sim: 0.7634

That’s retrieval. You got three functions, ranked by how close their embedding vectors are to the query vector. The scores tell you the retrieval worked. They tell you nothing about which lines in withRetry are relevant, whether RefreshToken is relevant for the same reason or a different one, or what “handle retries” actually means in context of each function body.

That gap — between “this is the right function” and “here is why it answers your question” — is the “R” in RAG. You can’t close it with better vectors. You close it by sending the retrieved code to a language model and asking it to explain.

Retrieval is not synthesis #

Most descriptions of RAG treat retrieval and synthesis as pipeline stages: chunk, embed, retrieve, generate. That framing is technically correct but obscures where the intelligence actually lives. In this series, retrieval has done a lot of work: enriched chunks (Part 7), hybrid search (Part 7), steering (Part 8). The quality of the synthesis depends entirely on retrieval quality. Feed the LLM good functions and it explains them well. Feed it noise and it summarises the noise confidently.

The LLM is the presentation layer. It is not the intelligence layer. If your retrieval is wrong, a better model will not fix it.

This part adds the presentation layer: a --explain flag on embed search that, for each retrieved result, streams an explanation of why that function answers the query.

What gets sent to the model #

Each retrieved document already has an enriched string stored in the index — the synthetic format from Part 7:

File: db/retry.go | Package: db | Function: withRetry | Body: func withRetry(ctx context.Context, maxAttempts int, fn func() error) error { ... }

That string is what we built the embedding from. It’s also what we send to the LLM. The file, package, and function name give the model enough context to be specific in its explanation — it can say “line 12 of withRetry in db/retry.go” rather than “the function you asked about.”

The prompt:

Given this Go function:

<function>
File: db/retry.go | Package: db | Function: withRetry | Body: func withRetry(...) { ... }
</function>

Explain how it answers the following question: "how does the app handle retries"

Be specific about which lines or blocks are directly relevant and why.
If the function is not relevant to the question, say so briefly.

One call per result. Three results, three calls, three streamed explanations printed beneath each similarity score.

The synthesizer package #

The shared/embedder package handles one operation: embedding text to a vector. Synthesis is a different operation — sending a chat prompt and streaming the response. It lives in its own synthesizer package alongside the other part-local packages.

// synthesizer/client.go

package synthesizer

// Client streams an explanation of how a code chunk answers a query.
type Client interface {
    Explain(ctx context.Context, chunk, query string) (io.Reader, error)
}

Explain returns an io.Reader rather than a string. That’s the streaming contract: the caller reads bytes as they arrive from the API, rather than waiting for the full response to buffer. For terminal output, this means the explanation appears word by word rather than after a noticeable pause.

Prompt construction #

// synthesizer/prompt.go

func buildPrompt(chunk, query string) string {
    return strings.Join([]string{
        "Given this Go function:",
        "",
        "<function>",
        chunk,
        "</function>",
        "",
        `Explain how it answers the following question: "` + query + `"`,
        "",
        "Be specific about which lines or blocks are directly relevant and why.",
        "If the function is not relevant to the question, say so briefly.",
    }, "\n")
}

OpenAI streaming implementation #

// synthesizer/openai.go

func (o *OpenAIClient) Explain(ctx context.Context, chunk, query string) (io.Reader, error) {
    prompt := buildPrompt(chunk, query)

    stream := o.client.Chat.Completions.NewStreaming(ctx, openai.ChatCompletionNewParams{
        Model: openai.ChatModel(o.model),
        Messages: []openai.ChatCompletionMessageParamUnion{
            openai.UserMessage(prompt),
        },
    })

    pr, pw := io.Pipe()

    go func() {
        defer pw.Close()
        for stream.Next() {
            event := stream.Current()
            if len(event.Choices) > 0 {
                delta := event.Choices[0].Delta.Content
                if delta != "" {
                    fmt.Fprint(pw, delta)
                }
            }
        }
        if err := stream.Err(); err != nil {
            pw.CloseWithError(err)
        }
    }()

    return pr, nil
}

The goroutine writes to a PipeWriter as tokens arrive; the PipeReader is returned immediately to the caller. This lets the search command print results incrementally without accumulating the full response.

Testing before the API #

The synthesizer is harder to unit test than the vector math — you can’t hand-craft an expected LLM response. What you can test is the contract:

// synthesizer/mock.go

type MockClient struct {
    Response string
}

func (m *MockClient) Explain(_ context.Context, _, _ string) (io.Reader, error) {
    return strings.NewReader(m.Response), nil
}

The mock satisfies the Client interface. Use it in cmd/ tests to verify that --explain reads from the reader and prints each explanation beneath its result — without hitting the API.

The prompt construction is a pure function and can be tested directly:

func TestBuildPrompt_ContainsChunk(t *testing.T) {
    chunk := "File: db/retry.go | Function: withRetry | Body: func withRetry() {}"
    query := "how does the app handle retries"
    prompt := buildPrompt(chunk, query)

    if !strings.Contains(prompt, chunk) {
        t.Error("prompt must contain the full chunk")
    }
}

The --explain flag #

One new flag on the existing search command. The rest of the search pipeline — indexing, enrichment, filtering, steering — runs as before. --explain only affects what happens after results are returned.

searchCmd.Flags().Bool("explain", false, "Stream an LLM explanation for each result")
searchCmd.Flags().String("chat-model", "gpt-4o-mini", "Chat model for synthesis (used with --explain)")

The synthesizer client is created only when --explain is set, using the same provider as the embedder:

var synth synthesizer.Client
if explain {
    switch provider {
    case "openai":
        synth = synthesizer.NewOpenAIClient(chatModel)
    case "ollama":
        synth = synthesizer.NewOllamaClient(chatModel)
    }
}

After each result is printed, printWithExplanation handles the output:

func printWithExplanation(
    ctx context.Context,
    synth synthesizer.Client,
    chunk, query string,
    rank int,
    file, fn string,
    score float32,
    explain bool,
) error {
    fmt.Printf("%d. %-36s :: %-20s sim: %.4f\n", rank, file, fn, score)

    if !explain || synth == nil {
        return nil
    }

    reader, err := synth.Explain(ctx, chunk, query)
    if err != nil {
        return fmt.Errorf("explain result %d: %w", rank, err)
    }

    fmt.Print("   > ")
    scanner := bufio.NewScanner(reader)
    scanner.Split(bufio.ScanRunes)
    for scanner.Scan() {
        text := scanner.Text()
        if text == "\n" {
            fmt.Print("\n   > ")
        } else {
            fmt.Print(text)
        }
    }
    fmt.Println()

    return scanner.Err()
}

The scanner reads one rune at a time and prefixes each newline with > to keep the explanation visually indented under its result. It’s a small formatting detail that matters when three explanations are streaming one after another.

Running it #

./embed search "how does the app handle retries" \
    --suppress "if err != nil" \
    --explain \
    --chat-model gpt-4o-mini \
    --top 3

Suppressing: "if err != nil" (weight 0.30)

1. db/retry.go      :: withRetry        sim: 0.7923
   > withRetry is directly relevant to this question. The function implements
   > exponential backoff retry logic: the for loop on lines 14–22 calls fn()
   > up to maxAttempts times, and on failure waits before the next attempt
   > using time.Sleep with a backoff duration. The context cancellation check
   > on line 18 ensures retries stop if the caller's context expires. This is
   > the app's primary mechanism for handling transient failures.

2. auth/token.go    :: RefreshToken     sim: 0.7711
   > RefreshToken is partially relevant. It retries the token exchange on a
   > 401 response (lines 31–35), but this is incidental retry logic embedded
   > in authentication flow rather than a general-purpose retry mechanism.
   > If you are looking for where retries are centrally managed, withRetry
   > in db/retry.go is the more direct answer.

3. cache/store.go   :: SetWithExpiry    sim: 0.7634
   > This function is not directly relevant to retry handling. It sets a
   > cache entry with a TTL and returns an error on failure, but does not
   > retry. Its similarity score likely reflects shared error-handling
   > vocabulary rather than retry logic.

The third result is an honest answer: the function is not relevant, and the model says so. That is more useful than a confident but inaccurate explanation.

What the model choice affects #

gpt-4o-mini is the default. It is fast, cheap, and accurate enough for code explanation. gpt-4o produces more precise line-level reasoning but costs roughly 10x more per token and streams noticeably slower.

For Ollama users, qwen2.5:latest works for synthesis the same way it works for embeddings. The response quality is lower on complex code, but for a local, free option that streams to your terminal it is adequate for exploration. The synthesizer follows the same interface pattern as the embedder: NewOllamaClient returns a synthesizer.Client. Swap the provider without changing any other code.

Getting started #

cd tutorial/part9/

complete/ has the working implementation. start/ has buildPrompt, Explain (in both OpenAI and Ollama clients), and printWithExplanation stubbed out. Run go test ./... to see which functions are failing; the mock client tests will pass, and the synthesizer tests will panic until buildPrompt and Explain are implemented.

export OPENAI_API_KEY="sk-your-key-here"

cd start/
go mod tidy
go build -o embed .
./embed search --index --enrich --provider openai --model text-embedding-3-small
./embed search "how does the app handle retries" --explain --top 3

What’s next #

The pipeline now retrieves relevant functions and explains them in context. What it still cannot see is the call graph. withRetry is useful because db/query.go calls it — but if you search for Execute, nothing in the retrieval pipeline surfaces that relationship. Part 10 adds that: static analysis of the Go AST to build a call graph alongside the vector index, so retrieved functions can be expanded by one hop to include their callers and callees.

The tutorial repository and all code: rikdc/semantic-search-experiments