The problem behind the interface
The source corpus contained roughly 150,000 classified website customer chats. Reviewers needed a practical way to investigate that corpus for customer-experience audits. A conversational interface could make the data accessible, but a plausible wrong answer would make the product less useful than the tools it replaced.
I built the backend around a different contract: the model can explain an answer, but the application owns the route to the facts.
Put the guarantees below the prompt
Claude, running on Amazon Bedrock, uses typed tools for aggregation, search, and examples. On the normal path, validated Python query builders produce the SQL. Curated DuckDB views encode exclusions and ranking rules so every request does not have to rediscover them.
Percentages carry their population in their names. Tool results include the rows, SQL, denominator, and exclusions. A deterministic verification step checks significant figures in the prose against the returned evidence. Findings are recorded for investigation; verification is not a claim that a language model can never make a mistake.
A guarded SQL escape hatch uses DuckDB’s parser to check the query structure. The data plane remains read-only, with external access disabled and a schema contract checked before serving requests.
Own the boundary, not just the endpoint
Agent frameworks create routes of their own. Authentication therefore sits in middleware, in front of the entire application. Deployed stages refuse to start with an open authentication configuration or missing required logging.
Query records in DynamoDB and traces in Langfuse make the path from question to answer inspectable. Tests assemble the real application and middleware, substituting only external dependencies. The build enforces a minimum of 80% branch coverage.
What this work represents
The customer-insights platform brings together Python API design, analytics semantics, production AI, identity, testing, and AWS delivery. The interface is conversational. The important work is giving that conversation a dependable foundation.