How Shopify Scaled AI-Powered Product Recommendations to 10M Daily Users
How Shopify Scaled AI-Powered Product Recommendations to 10M Daily Users
Published: June 2026 | Reading time: 12 min | Category: Case Studies
The Business Problem
Shopify’s merchants were losing an estimated $2.3B annually in abandoned carts driven by irrelevant product recommendations. The legacy system — a basic collaborative-filtering engine built in 2019 — had hit its ceiling:
- 78% of „recommended“ products were already in the customer’s cart
- Average recommendation click-through rate had dropped to 1.2%
- The system couldn’t incorporate product images, descriptions, or real-time behavior
- Cold-start problem: new products and new users got terrible recommendations
The Solution Architecture
The new system, codenamed Mercury, is a three-stage recommendation pipeline:
Stage 1: Candidate Generation (milliseconds)
├── Embedding-based retrieval (FAISS + product/user embeddings)
├── Real-time session signals (last 5 actions via Kafka)
├── Output: 1,000 candidate products from catalog of 50M+
Stage 2: Ranking Model (~50ms)
├── Multi-task model: predicts CTR, add-to-cart, and purchase
├── Features: user history, product attributes, session context, inventory
├── Transformer architecture with cross-attention between user and product
├── Output: Ranked list of 100 products
Stage 3: Business Logic Layer (~10ms)
├── Inventory filtering (no out-of-stock recommendations)
├── Diversity injection (avoid showing 10 similar items)
├── Business rules (promote new products, seasonal items)
├── A/B test assignment
└── Output: Final 4-8 products shown to user
The Training Data Pipeline
| Data Source | Volume | Freshness | Purpose |
|---|---|---|---|
| Clickstream events | 500M/day | Real-time (Kafka) | Session context features |
| Purchase history | 50M orders | Hourly batch | Long-term user preferences |
| Product catalog | 50M products | Near real-time | Product embedding updates |
| Merchant metadata | 2M merchants | Daily | Business rule configuration |
Key Technical Decisions
Decision 1: Real-Time vs Batch Features
The team made a crucial decision to compute user embeddings in real-time rather than batch. Previous systems had a 24-hour staleness problem — a user who searched for „running shoes“ at 9 AM wouldn’t see shoe recommendations until the next day.
Solution: A Flink-based streaming pipeline that updates user embeddings every 5 minutes based on session activity, backed by a Redis cache for sub-millisecond reads.
Decision 2: Multi-Modal Product Understanding
Instead of relying on product IDs and categorical features, Mercury uses a vision-language model to generate product embeddings from images and descriptions:
product_embedding = clip_model.encode(
image=product.thumbnail_image,
)
# 1,034-dim embedding that captures visual + semantic similarity
This enabled „visually similar“ recommendations and dramatically improved cold-start performance for new products.
Decision 3: Edge vs Cloud Inference
Initially, all ranking happened in the cloud. P99 latency was 340ms — too slow for the „Related Products“ widget that loads on every product page.
Solution: A two-tier approach. Candidate generation and the first ranking pass run on edge servers (< 50ms). Final ranking runs in the cloud (additional 50ms). This brought P99 latency to 85ms.
Results at Scale
| Metric | Legacy System | Mercury | Improvement |
|---|---|---|---|
| CTR on recommendations | 1.2% | 4.7% | +291% |
| Revenue per session | $0.12 | $0.38 | +217% |
| Cold-start product discovery | 0.3% CTR | 2.1% CTR | +600% |
| Infrastructure cost/month | $45K | $120K | +167% |
| P99 latency | 450ms | 85ms | -81% |
Net result: 3.2x revenue increase at 2.7x infrastructure cost — a clear win. Annual revenue attribution from Mercury: $89M incremental GMV.
1. The first launch increased infrastructure costs by 400% before optimization. The team underestimated the serving cost of real-time features.
2. Early versions showed strong filter bubbles — users only saw variations of what they’d already bought. The diversity injection module was added 6 months late.
3. The multi-modal product embeddings had a bias problem: products with professional photography were systematically recommended over products with amateur photos, regardless of quality.
Lessons for Your Implementation
- Start with candidate generation, not ranking: Getting the right 1,000 candidates matters more than perfectly ranking the top 10. Invest in your retrieval layer first.
- Real-time features aren’t optional in 2026: Batch-only user features mean stale recommendations. Budget for streaming infrastructure from day one.
- Plan for the cold-start problem: New products and users are your growth engine. Multi-modal embeddings (images + text) solve this elegantly.
- Budget 6 months for fairness and diversity: Unchecked recommendation systems create filter bubbles. Build diversity and fairness constraints into your system from the start.
What Shopify Is Building Next
The next version of Mercury, currently in testing, uses an LLM-based „recommendation explainer“ — instead of just showing products, it generates natural language explanations: „We recommend these running shoes because you loved the Nike Pegasus and this model has similar cushioning technology at a lower price point.“
Early testing shows a 23% increase in CTR when explanations are included, though the added latency (200ms for LLM generation) remains a challenge.
Related: RAG Architecture Patterns | AI Inference Optimization | Content Hub
Schreibe einen Kommentar