body { font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, sans-serif; max-width: 800px; margin: 0 auto; padding: 20px; line-height: 1.7; color: #1a1a2e; }
h1 { color: #16213e; border-bottom: 3px solid #059669; padding-bottom: 10px; }
h2 { color: #0f3460; margin-top: 30px; }
h3 { color: #533483; }
.highlight { background: #ecfdf5; border-left: 4px solid #059669; padding: 15px; margin: 15px 0; border-radius: 0 8px 8px 0; }
.warning { background: #fff3f3; border-left: 4px solid #e94560; padding: 15px; margin: 15px 0; border-radius: 0 8px 8px 0; }
.code { background: #1e1e2e; color: #cdd6f4; padding: 15px; border-radius: 8px; font-family: 'Fira Code', monospace; overflow-x: auto; }
.keyword { color: #059669; font-weight: bold; }
table { border-collapse: collapse; width: 100%; margin: 20px 0; }
th { background: #16213e; color: white; padding: 12px; text-align: left; }
td { border: 1px solid #ddd; padding: 10px; }
tr:nth-child(even) { background: #f8f9fa; }

return escalate_to_human()

return answer

Building a Production RAG System: How Stripe Reduced Support Ticket Volume by 40%

Q: The Chunking Strategy That Actually Works

Stripe's first attempt used fixed-size 512-token chunks. It failed badly — code examples were split mid-function, and FAQ answers were fragmented across multiple chunks. Their solution: semantic chunking with source-aware boundaries: Source TypeChunking StrategyAvg Chunk Size API docsSection boundar

Q: Handling the Hallucination Problem

The biggest challenge wasn't retrieval quality — it was the LLM confidently generating incorrect API code. Early versions would: Invent non-existent API parameters Mix code from different API versions Generate syntactically valid but semantically wrong code Stripe's solution was a code verification

Q: Results After 18 Months

MetricBefore RAGAfter RAGChange Tickets handled autonomously0%40%+40pp Autonomous resolution rateN/A94%— Avg response time4.2 hours8 seconds-99.9% Customer satisfaction (CSAT)3.8/54.3/5+13% Human agent capacity50K tickets/mo30K tick

Q: Key Implementation Patterns

1. Hybrid Search is Non-Negotiable Pure semantic search misses exact matches (error codes, API parameter names). Pure keyword search misses conceptual questions. Stripe uses a weighted combination: 70% dense (embedding) + 30% sparse (BM25), merged with Reciprocal Rank Fusion. 2. Citation Requirement

Published: June 2026 | Reading time: 13 min | Category: Case Studies

The story: Stripe’s developer support team was drowning in repetitive questions. They built a RAG-powered support assistant that now handles 40% of all support tickets autonomously, with a 94% resolution rate. Here’s the full architecture and the hard-won lessons from running it in production for 18 months.

The Problem: Support at Scale

Stripe processes payments for millions of businesses. Their developer support team handles ~50,000 tickets per month, and analysis showed that 60% of tickets were variations of the same ~200 questions:

„How do I set up webhooks?“
„What’s the difference between PaymentIntents and Charges?“
„Why is my test API key returning 401 errors?“
„How do I handle 3D Secure authentication?“

Human agents were spending 70% of their time on questions that had well-documented answers. The remaining 30% — complex, novel issues — weren’t getting enough attention.

The RAG Architecture

# Stripe Support RAG Pipeline

Knowledge Sources:
├── API Documentation (15,000 pages, Markdown)
├── Support Ticket History (2M resolved tickets)
├── GitHub Issues & Discussions (50K threads)
├── Internal Runbooks (200 documents)
└── Changelog & Migration Guides (500 docs)

Processing Pipeline:
1. Document Ingestion (hourly)
├── Scrape & parse all sources
├── Chunk with semantic boundaries (512 tokens)
├── Generate embeddings (text-embedding-3-large)
└── Store in Pinecone (10M+ vectors)

2. Query Processing (real-time)
├── User question → embedding
├── Hybrid search (dense + BM25)
├── Rerank with cross-encoder
├── Top-5 chunks → context window
└── LLM generates answer with citations

3. Quality Assurance
├── Confidence scoring (reject if < 0.7)
├── Citation verification (all claims must have sources)
├── PII detection (redact before showing)
└── Escalation routing (if confidence low)

The Chunking Strategy That Actually Works

Stripe’s first attempt used fixed-size 512-token chunks. It failed badly — code examples were split mid-function, and FAQ answers were fragmented across multiple chunks.

Their solution: semantic chunking with source-aware boundaries:

Source Type	Chunking Strategy	Avg Chunk Size
API docs	Section boundaries (headers)	300-800 tokens
Code examples	Function/class boundaries	200-500 tokens
FAQ/tickets	Question-answer pairs	150-400 tokens
Runbooks	Step-by-step procedures	400-700 tokens

They also added metadata-rich chunk headers to every chunk:

# Chunk metadata format
{
„source“: „api_docs/payment_intents/create“,
„section“: „Creating a PaymentIntent“,
„doc_version“: „2024-06-20“,
„chunk_type“: „code_example“, # or „explanation“, „reference“
„api_version“: „2024-06-20“,
„products“: [„payments“, „connect“]
}

This metadata enables filtered retrieval — when a user asks about „PaymentIntents,“ the system only searches chunks tagged with that product, dramatically improving relevance.

Handling the Hallucination Problem

The biggest challenge wasn’t retrieval quality — it was the LLM confidently generating incorrect API code. Early versions would:

Invent non-existent API parameters
Mix code from different API versions
Generate syntactically valid but semantically wrong code

Stripe’s solution was a code verification layer:

# Code verification pipeline

def verify_response(answer, retrieved_chunks):

# 1. Extract all code blocks from the answer
code_blocks = extract_code(answer)

for code in code_blocks:

# 2. Check all API calls against the retrieved documentation
api_calls = parse_api_calls(code)

for call in api_calls:
if not is_documented(call, retrieved_chunks):

flag_as_potential_hallucination(call)

# 3. Run syntax check (for supported languages)

if not syntax_check(code):

flag_as_invalid(code)

# 4. If any flags, either regenerate or escalate

if has_flags():
if confidence > 0.8:
return regenerate_with_warning(flags)

else:

Results After 18 Months

Metric	Before RAG	After RAG	Change
Tickets handled autonomously	0%	40%	+40pp
Autonomous resolution rate	N/A	94%	—
Avg response time	4.2 hours	8 seconds	-99.9%
Customer satisfaction (CSAT)	3.8/5	4.3/5	+13%
Human agent capacity	50K tickets/mo	30K tickets/mo + complex work	+quality
Monthly infrastructure cost	$0	$12,000	—

What went wrong:

1. The system initially couldn’t handle multi-turn conversations. Users would ask follow-up questions and the system would lose context. Fixed by adding conversation memory and re-retrieving on each turn.

2. Version confusion: the system would mix documentation from different API versions. Fixed by adding version metadata and filtering by the user’s declared API version.

3. Overconfidence: the system would answer questions outside its knowledge base with plausible-sounding but wrong answers. Fixed by implementing the confidence threshold and escalation routing.

Key Implementation Patterns

1. Hybrid Search is Non-Negotiable

Pure semantic search misses exact matches (error codes, API parameter names). Pure keyword search misses conceptual questions. Stripe uses a weighted combination: 70% dense (embedding) + 30% sparse (BM25), merged with Reciprocal Rank Fusion.

2. Citation Requirements Improve Quality

Requiring the LLM to cite sources for every claim reduced hallucination by 60%. The model can’t make up API parameters if it has to point to the documentation that defines them.

3. Graceful Escalation is a Feature

The system confidently says „I don’t know“ and escalates to a human when confidence is below threshold. This is better than a wrong answer. Stripe’s escalation rate is 8% — and those tickets get priority routing to senior agents.

Key takeaway: RAG systems in production are 20% model and 80% engineering. The quality of your chunking, metadata, retrieval strategy, and verification layers matters more than which LLM you choose. Stripe’s 94% resolution rate came from 18 months of iterative improvement to the pipeline — not from a single model upgrade.