--- title: "Real-life scenario: ClaimsPilot" chapter: "23" --- # Real-life scenario: ClaimsPilot ClaimsPilot helps insurance adjusters review a claim packet, find policy evidence, detect missing information, draft a recommendation, and prepare approved actions. It never makes or pays a final claim autonomously. ## User outcome An adjuster uploads forms, photos, repair estimates, emails, and a policy ID. Within two minutes the workspace should: - extract typed facts with source coordinates; - retrieve current policy clauses valid on the loss date; - identify conflicts and missing evidence; - draft coverage reasoning with claim-level citations; - call read-only tools for claim/account state; - propose—not execute—reserve or communication changes; - route uncertainty and high-risk cases to a human. ## Architecture `Angular UI → Spring Boot 4 / Spring AI 2 → policy/authorization → document pipeline → pgvector + lexical search → reranker → model router → claim tools → schema/grounding checks → approval queue → audit` OpenAI GPT-5.6 Sol handles the hardest reasoned draft when approved for the data class. Gemini 3.6 Flash handles multimodal packet understanding and routine interaction. Ollama handles private preliminary extraction and an independent groundedness checker. These are evaluated roles, not automatic fallbacks. ## Ingestion Files are malware-scanned, authorized, parsed/OCRed, and split by form section, policy clause, or image/page. Every chunk stores claim/tenant, ACL, source hash, page/coordinates, loss-date applicability, parser version, embedding identity, and retention key. Policy material enters a separate versioned corpus. An effective-date filter is mandatory. Old policy wording is retained for claims arising under it. ## Retrieval The system extracts exact policy/form identifiers for lexical retrieval and embeds the natural-language question for semantic retrieval. It fuses, reranks, and rejects low relevance. Citations are created from metadata and validated against returned evidence. ## Spring AI implementation ```java record ClaimDraft( List facts, List reasons, List missing, ProposedAction proposedAction, boolean humanReviewRequired) {} ClaimDraft draft = claimsClient.prompt() .advisors(claimMemory, policyRag, auditAdvisor) .tools(new ClaimReadTools(claimService, authorization)) .system(CLAIMS_POLICY) .user(packetQuestion) .call() .entity(ClaimDraft.class); validator.validateSchema(draft); grounding.validateEveryCitation(draft); policyEngine.validateProposal(user, claim, draft.proposedAction()); approvalQueue.submit(draft); // no payment or customer message yet ``` `ToolCallingAdvisor` handles the bounded read-only loop. Write tools are absent. After an adjuster approves an exact proposal, a separate normal REST command performs authorization, optimistic locking, idempotency, and audit. ## Failure paths - no policy clause: abstain and request a policy specialist; - conflicting OCR: show both sources and request confirmation; - stale claim version: return conflict and rebuild the proposal; - provider timeout: save resumable job state; do not claim success; - malicious instruction inside email: treat as quoted evidence, never a command; - low retrieval score: no-answer state, not a creative guess; - model/schema failure: one bounded correction, then human queue; - tool unavailable: state the missing fact and avoid inventing it. ## Evaluation and rollout The golden set includes routine, ambiguous, denied, fraud-sensitive, multilingual, scanned, conflicting, injection, and cross-tenant cases. Release gates require zero unauthorized tool/data access, complete valid citations, high policy-clause recall, bounded latency/cost, and no regression by claim slice. Shadow evaluation uses redacted approved cases. Canary begins with read-only drafts for trained adjusters. Expansion requires measured correction rate, groundedness, time saved, escalation quality, and incident-free evidence. ## Glossary and abbreviations | Term | Plain meaning | |---|---| | AI / ML | Artificial intelligence / machine learning | | LLM / VLM | Language model / vision-language model | | Token / context window | Text piece / maximum working input-output space | | Inference | Running fixed model weights to produce output | | RAG | Retrieve evidence, then generate with it | | Embedding | Numeric coordinate representing content meaning | | ANN / HNSW / IVFFlat | Approximate search / two vector index approaches | | BM25 / hybrid search | Lexical ranking / lexical plus vector retrieval | | Reranker | More precise second-stage candidate scorer | | SFT / DPO / RFT | Supervised / preference / reinforcement fine-tuning | | LoRA / QLoRA | Small trainable adapter / quantized adapter training | | Quantization | Smaller numeric weights for cheaper inference | | Tool calling | Model requests a typed application function | | MCP | Model Context Protocol for tools, resources, and prompts | | Agent | Bounded model-driven observe/action loop | | Advisor | Spring AI interceptor/orchestration component | | ETL | Extract, transform, load document pipeline | | ACL / RBAC | Access list / role-based access control | | PII | Personally identifiable information | | MRR / nDCG | Retrieval ranking quality metrics | | Groundedness | Claims supported by supplied evidence | | Hallucination | Unsupported model-generated content | | Golden set | Stable labeled evaluation examples | | Shadow / canary | Compare invisibly / expose to a controlled slice | | TTFT | Time to first token | | KV cache | Reused transformer attention state | | GPU / VRAM | Accelerator / its working memory | ## Official references - [Spring AI 2.0 GA](https://spring.io/blog/2026/06/12/spring-ai-2-0-0-GA-available-now/) - [Spring AI reference](https://docs.spring.io/spring-ai/reference/) - [Spring AI RAG](https://docs.spring.io/spring-ai/reference/api/retrieval-augmented-generation.html) - [Spring AI tools](https://docs.spring.io/spring-ai/reference/api/tools.html) - [OpenAI model guidance](https://developers.openai.com/api/docs/guides/latest-model) - [OpenAI tools](https://developers.openai.com/api/docs/guides/tools) - [OpenAI embeddings](https://developers.openai.com/api/docs/guides/embeddings) - [OpenAI model optimization](https://developers.openai.com/api/docs/guides/model-optimization) - [Gemini models](https://ai.google.dev/gemini-api/docs/models) - [Gemini embeddings](https://ai.google.dev/gemini-api/docs/embeddings) - [Gemini tools](https://ai.google.dev/gemini-api/docs/tools) - [Gemini changelog](https://ai.google.dev/gemini-api/docs/changelog) - [Ollama documentation](https://docs.ollama.com/) - [Ollama embeddings](https://docs.ollama.com/capabilities/embeddings) - [Ollama model import](https://docs.ollama.com/import) ## Final Feynman challenge Trace one ClaimsPilot question from authenticated user, ingestion and ACL, through hybrid retrieval, embeddings, reranking, prompt assembly, model route, tool call, structured draft, citation validation, approval, command, observability, evaluation, canary, and rollback.