URL Shortener: The "Simple" System That's Anything But
Everyone thinks a URL shortener is the "Hello World" of system design interviews. Hash a URL, store the mapping, redirect on lookup — done, right? Not even close. Bitly handles 600 million clicks per month. TinyURL processes millions of URLs daily. When you dig in, you're designing a globally distributed, read-heavy, analytics-powered system that needs to respond in single-digit milliseconds.
Why It's Harder Than It Looks
The core challenge is the traffic pattern. For every URL you shorten, it might get clicked 100 times. That means you're building a system that's 99% reads — but still needs to handle millions of writes per day. Add analytics (clicks by country, device, time), abuse prevention, and global latency requirements, and your "simple" project is suddenly a real engineering challenge.
Quick math at scale: 100M URLs shortened daily × 100 clicks each = 10 billion redirects/day. That's ~115,000 reads per second and 1,200 writes per second. Storage for 5 years? About 90TB just for URL mappings.
The Encoding Problem: How to Generate Short Codes
This is where most candidates stumble. Here are your options:
| Approach | Pros | Cons | Verdict |
|---|---|---|---|
| MD5/SHA + Truncate | Simple | Unpredictable collisions, no fix without retry | ❌ Avoid |
| Auto-increment Counter | No collisions, simple | Predictable (scraping risk), single point of failure | ⚠️ Risky alone |
| Base62 + Counter | Compact, no collisions | Still predictable unless shuffled | ✅ With randomization |
| Snowflake-style ID | Unique, distributed, not predictable | More complex, longer codes | ✅ Best at scale |
The Instagram approach is elegant: combine a timestamp, machine ID, and sequence number, then base62 encode the whole thing. You get uniqueness without a single point of coordination, and the codes aren't predictable enough to scrape.
Key insight: Hashing is a trap. You can't efficiently detect or resolve collisions at scale. Counter-based approaches with distributed coordination (like Snowflake IDs) are what production systems actually use.
The Interview Framework
When you get this question, follow this path:
- Functional requirements — Shorten URLs, redirect (301 vs 302?), custom aliases, expiration, analytics
- Scale estimation — Write QPS, read QPS, storage over N years, bandwidth
- High-level architecture — Load balancer → Web servers → Encoding service → Database + Cache
- Deep dive on encoding — How to generate short codes without collisions
- Caching strategy — The 100:1 read ratio demands aggressive caching
- Extensions — Analytics, rate limiting, geographic distribution
The 301 vs 302 redirect choice matters more than you'd think: 301 (permanent) is better for SEO but makes analytics harder because browsers cache it. 302 (temporary) ensures every click hits your server — critical for accurate tracking.
Caching: Your Most Important Decision
With a 100:1 read-to-write ratio, your caching layer is arguably more important than your database choice. The pattern is straightforward:
Read path: Check cache → hit? redirect immediately. Miss? query DB → populate cache → redirect.
What to cache: The top 20% of URLs (by popularity) will cover 80%+ of your traffic. Use an LRU eviction policy with TTLs. A few hundred GB of Redis handles billions of active mappings.
Cache invalidation: Mostly a non-issue for URL shorteners — mappings rarely change. Expired URLs get evicted naturally. This is one of the rare systems where caching is simple because the data is essentially immutable.
Database Design: SQL vs NoSQL
For the URL mapping itself (short_code → long_url), you need:
- Fast point lookups by short_code (the redirect hot path)
- Fast writes when creating new URLs
- Horizontal scalability to billions of rows
NoSQL (DynamoDB, Cassandra) is the natural fit: key-value lookups are the primary access pattern, and sharding by short_code distributes load evenly. No joins needed.
SQL (MySQL, PostgreSQL) works too with proper indexing and sharding, especially if you want ACID guarantees for deduplication or custom alias reservation.
Sharding strategy: Hash the short_code to determine the shard. Since codes are already pseudo-random (if using Snowflake-style IDs), you get even distribution for free.
Real-World Implementations
Bitly evolved from a single MySQL instance to a distributed architecture. Their key insight: separate the write path (URL creation) from the read path (redirects) entirely. Writes go to a primary database; reads are served from replicas and cache.
TinyURL kept it simple — a focused architecture that proves you don't always need microservices. Their success shows that a well-tuned monolith with good caching can handle enormous scale.
Instagram's ID generation solved the distributed uniqueness problem without coordination: each shard generates its own IDs using (timestamp, shard_id, sequence). This pattern is directly applicable to URL shortener code generation.
Key Takeaways
- It's a read-heavy system — Design for the 100:1 ratio from day one. Cache aggressively.
- Don't hash for short codes — Use counter-based or Snowflake-style distributed ID generation
- 301 vs 302 is a product decision — Analytics needs vs SEO benefits
- Separate read and write paths — They have completely different scale characteristics
- Caching is your biggest lever — A few hundred GB of Redis eliminates most database load
- Shard by short_code — Even distribution, fast lookups, scales horizontally
URL Shortener：看似简单，实则不然
很多人觉得 URL Shortener 就是系统设计面试里的 "Hello World"。哈希一下 URL，存个映射，查到了就重定向——搞定了？远不止如此。Bitly 每月处理 6 亿次点击，TinyURL 每天处理数百万条 URL。深入一看，你其实在设计一个全球分布式、读密集型、带分析能力的系统，而且要求毫秒级响应。
为什么比想象中难
核心挑战在于流量模式。每缩短一条 URL，它可能会被点击 100 次。这意味着你要构建一个 99% 都是读的系统——但每天仍然要处理数百万次写入。再加上分析（按国家、设备、时间统计点击量）、防滥用和全球低延迟要求，你的 "简单" 项目瞬间变成了一个真正的工程挑战。
规模速算： 每日 1 亿条 URL 缩短 × 每条 100 次点击 = 每天 100 亿次重定向。相当于约 115,000 次读/秒和 1,200 次写/秒。存储 5 年？仅 URL 映射就约 90TB。
编码问题：如何生成短码
这是大多数候选人卡壳的地方。你的选项如下：
| 方案 | 优点 | 缺点 | 结论 |
|---|---|---|---|
| MD5/SHA + 截断 | 简单 | 碰撞不可预测，除了重试没有修复手段 | ❌ 避免 |
| 自增计数器 | 无碰撞，简单 | 可预测（爬虫风险），单点故障 | ⚠️ 单独使用有风险 |
| Base62 + 计数器 | 紧凑，无碰撞 | 不做混淆则仍可预测 | ✅ 配合随机化使用 |
| Snowflake 式 ID | 唯一、分布式、不可预测 | 更复杂，码更长 | ✅ 大规模最佳选择 |
Instagram 的方案很优雅：将时间戳、机器 ID 和序列号组合，然后对整体做 Base62 编码。你无需单点协调就能获得唯一性，而且码不容易被预测到可以批量爬取。
关键洞察： 哈希是个陷阱。在大规模下你无法高效检测或解决碰撞。基于计数器加分布式协调（如 Snowflake ID）的方案才是生产系统实际使用的方式。
面试框架
当你遇到这个题时，按这个路径走：
- 功能需求 — 缩短 URL、重定向（301 还是 302？）、自定义别名、过期机制、分析
- 规模估算 — 写 QPS、读 QPS、N 年的存储量、带宽
- 高层架构 — 负载均衡 → Web 服务器 → 编码服务 → 数据库 + 缓存
- 深入编码方案 — 如何无碰撞地生成短码
- 缓存策略 — 100:1 的读比率要求激进缓存
- 扩展 — 分析、限流、地理分布
301 vs 302 重定向的选择比你想的重要：301（永久）对 SEO 更好，但浏览器会缓存，导致分析数据不准。302（临时）确保每次点击都到达你的服务器——对精确追踪至关重要。
缓存：你最重要的决策
100:1 的读写比意味着你的缓存层可以说比数据库选型更重要。模式很直接：
读路径： 查缓存 → 命中？直接重定向。未命中？查 DB → 填充缓存 → 重定向。
缓存什么： 热度 top 20% 的 URL 会覆盖 80%+ 的流量。使用带 TTL 的 LRU 淘汰策略。几百 GB 的 Redis 就能处理数十亿条活跃映射。
缓存失效： 对 URL Shortener 来说基本不是问题——映射几乎不变。过期的 URL 自然被淘汰。这是少数缓存很简单的系统之一，因为数据本质上是不可变的。
数据库设计：SQL vs NoSQL
对于 URL 映射本身（short_code → long_url），你需要：
- 按 short_code 快速点查（重定向热路径）
- 创建新 URL 时的快速写入
- 水平扩展到数十亿行
NoSQL（DynamoDB、Cassandra） 是天然之选：主要访问模式就是键值查询，按 short_code 分片可以均匀分配负载。不需要 join。
SQL（MySQL、PostgreSQL） 在适当索引和分片下也可以工作，尤其当你需要 ACID 保证来做去重或自定义别名预留时。
分片策略： 对 short_code 做哈希来确定分片。如果使用 Snowflake 式 ID，码本身已经是伪随机的，自然获得均匀分布。
真实案例
Bitly 从单台 MySQL 实例演进到分布式架构。他们的关键洞察：将写路径（URL 创建）和读路径（重定向）完全分离。写入到主库；读取由副本和缓存提供服务。
TinyURL 保持简单——一个专注的架构证明了你并不总需要微服务。他们的成功表明，一个调优良好的单体配合优秀的缓存可以处理巨大的规模。
Instagram 的 ID 生成 在无需协调的情况下解决了分布式唯一性问题：每个分片使用（时间戳、shard_id、序列号）生成自己的 ID。这个模式直接适用于 URL Shortener 的短码生成。
核心要点
- 这是一个读密集系统 — 从一开始就为 100:1 的比率设计。激进缓存。
- 不要用哈希来生成短码 — 使用基于计数器或 Snowflake 式的分布式 ID 生成
- 301 vs 302 是产品决策 — 分析需求 vs SEO 收益
- 分离读写路径 — 它们有完全不同的规模特征
- 缓存是你最大的杠杆 — 几百 GB 的 Redis 就能消除大部分数据库负载
- 按 short_code 分片 — 均匀分布、快速查询、水平扩展