ChatClient, prompts, streaming, and structured output

ChatClient is the normal application boundary. Create role-specific clients rather than one global client with every advisor and tool.

record Answer(String summary, List<String> citations, double confidence) {}

Answer answer = chatClient.prompt()
    .system(s -> s.text(SUPPORT_POLICY).param("region", region))
    .user(question)
    .call()
    .entity(Answer.class);

Prompt contract

An effective prompt states:

• user-visible outcome;
• evidence and business constraints;
• available tools and when to use them;
• output schema;
• stopping and abstention conditions.

Keep system/developer policy stable and put changing user data later. Avoid contradictory absolute rules and repeated examples. Treat prompt templates as versioned code with tests and ownership.

Structured output

Use schema-constrained provider output where supported and validate the decoded Java record. Spring AI 2.0 supports provider-native structured output and self-correcting schema validation. A valid JSON shape can still contain an invalid business decision, so run domain validation afterward.

Streaming

Streaming improves perceived latency but complicates moderation, citations, tool loops, cancellation, retries, and UI recovery. Never repeat already displayed tokens after reconnect. A streamed partial sentence is not a committed business result.

Timeouts and cancellation

Set connect, read, overall-request, and tool-loop deadlines. Propagate client cancellation. Do not automatically retry a request after a tool may have performed a non-idempotent action.

Feynman check

The prompt is a job contract; structured output is a form; validation is the supervisor checking the filled form before software trusts it.