🏗️ 系统设计 / System Design — Day 77

设计 AI 聊天平台（ChatGPT 量级）/ Design an AI Chatbot Platform (ChatGPT-scale)

⏱️ 阅读时间约 3 分钟 / ~3 min read

🎬 场景导入 / Real-World Scenario

想象你在 OpenAI 的基础设施团队。产品刚刚宣布 ChatGPT 对所有人免费开放——你的系统需要在几分钟内从 100 万用户扩展到 1 亿用户。每个请求都是长连接 streaming，每个 token 都要实时推送。

Imagine you're on OpenAI's infra team. ChatGPT just went free for everyone — your system needs to scale from 1M to 100M users in minutes. Every request is a long-lived streaming connection, every token pushed in real time.

🏛️ ASCII 架构图 / Architecture

⚖️ 核心权衡 / Key Tradeoffs

为什么用队列而不是直接调 GPU？/ Why queue instead of direct GPU call?

• 直接调用：延迟低，但 GPU 过载时请求直接失败
• 队列：增加~50ms 延迟，但提供背压（backpressure）保护，允许优先级调度（付费用户 > 免费用户）
• Direct: lower latency but fails under load; Queue: ~50ms overhead but enables priority scheduling

流式输出 SSE vs WebSocket？/ SSE vs WebSocket?

• ChatGPT 用 SSE：单向、HTTP 友好、CDN 可缓存心跳、浏览器原生支持
• WebSocket 适合双向实时（游戏、协作编辑）；纯 LLM 流式输出 SSE 足够
• ChatGPT uses SSE: unidirectional, HTTP-friendly, no upgrade handshake

KV Cache 共享 / Shared KV Cache:

• vLLM 的 PagedAttention 允许多请求共享 prompt prefix 的 KV cache，显著提升 throughput
• vLLM's PagedAttention shares prompt prefix KV cache across requests — critical for system prompts

🚫 别踩这个坑 / Common Mistakes

1. 把对话历史塞进 context 而不截断 — 超出 context window，直接 crash。正确做法：滑动窗口 + 向量摘要

Don't stuff all history into context — sliding window + vector summary for old turns

1. 单点 GPU 节点不做健康检查 — GPU OOM 是常态，必须有自动剔除 + 重新路由

GPU OOM is routine — must auto-remove unhealthy nodes and reroute inflight requests

1. 忽略 cold start — 推理服务启动加载模型要 30-60s，必须预热（keep warm instances）

Model loading takes 30-60s — keep warm instances, don't scale to zero for LLM

1. Rate limiting 只在 API Gateway 做 — 绕过 Gateway 的内部流量会打穿 GPU

Rate limit at GPU cluster level too, not just the gateway

📚 References

• How ChatGPT is built — OpenAI engineering blog
• vLLM: Easy, Fast, and Cheap LLM Serving
• Server-Sent Events vs WebSocket

🧒 ELI5

把它想成一家超级繁忙的餐厅：API Gateway 是前台收银，Kafka 队列是点单系统，GPU 集群是厨房（很贵，每个厨师一秒钟能出几道菜），SSE 是服务员一道一道把菜端出来的过程。餐厅的关键不是厨师有多快，是怎么让厨师一直在干活不空等。

Think of it as a restaurant: the gateway is the cashier, the queue is the order system, GPUs are the (very expensive) chefs, and SSE is the waiter delivering dishes one by one. The key isn't chef speed — it's keeping chefs busy with zero idle time.

💻 算法 / Algorithms — Day 77

#211 Design Add and Search Words Data Structure (Medium)

⏱️ 阅读时间约 4 分钟 / ~4 min read

🧩 前缀树模式 (2/3) — 基于 Day 74 的 Trie 模板继续深化

Building on the Trie template from Day 74 (#208 Implement Trie)

🔗 Links

• 🔗 LeetCode #211 🟡 Medium
• 📹 NeetCode Video
• 🧩 Trie Pattern Block: #208 Implement Trie → #211 This Problem → #212 Word Search II (Hard)

🌍 现实类比 / Real-World Analogy

你在写一个 grep 工具，支持 . 通配符：se.rch 能匹配 search 也能匹配 seerch。字典里有几十万个单词，怎么做到 O(L) 匹配？

You're building grep with . wildcard: se.rch matches search or seerch. With 500K words in the dictionary, how do you match in O(L)?

🧩 与模板的关系 / How It Maps to the Template

上次的模板（#208）：insert + startsWith + search（精确匹配）

今天的变化（#211）：search 支持 . 通配符 → 需要DFS 回溯

Day 74's template: exact insert + search

Today's twist: search("b.d") must try ALL children when . is hit → DFS backtracking

模板核心变化 / Key Template Diff:

精确匹配 / Exact:     node = node.children[ch]   (直接跳 / direct jump)
通配匹配 / Wildcard:  for child in node.children.values():  (遍历全部 / try all)
                          dfs(child, i+1)

💻 Python 解法 + 追踪 / Solution with Trace

class TrieNode:
    def __init__(self):
        self.children = {}
        self.is_end = False

class WordDictionary:
    def __init__(self):
        self.root = TrieNode()

    def addWord(self, word: str) -> None:
        # Same as Day 74's insert template — unchanged
        node = self.root
        for ch in word:
            if ch not in node.children:
                node.children[ch] = TrieNode()
            node = node.children[ch]
        node.is_end = True

    def search(self, word: str) -> bool:
        # THE KEY CHANGE: DFS when we hit '.'
        def dfs(node: TrieNode, i: int) -> bool:
            if i == len(word):
                return node.is_end
            ch = word[i]
            if ch == '.':
                # Try ALL children — backtracking
                for child in node.children.values():
                    if dfs(child, i + 1):
                        return True
                return False
            else:
                # Exact match — O(1) lookup
                if ch not in node.children:
                    return False
                return dfs(node.children[ch], i + 1)

        return dfs(self.root, 0)

追踪 / Trace: 插入 "bad", "dad", "mad"，搜索 ".ad"

📊 复杂度 / Complexity

实际中 / In practice: most queries have few . → fast in practice, but O(N) worst case is real for "..." queries.

举一反三 / Pattern Family

核心洞察 / Insight: 每次 Trie 遇到"模糊"条件（通配符/近似匹配），精确查找就变成 DFS。理解这个跳跃就理解了 Trie 的全部变种。

Whenever a Trie query becomes "fuzzy," exact lookup becomes DFS. Master this transition and you understand all Trie variants.

📚 References

• LeetCode #211 editorial
• NeetCode Trie playlist

🧒 ELI5

普通 Trie 就像走一条有标记的路，每个字母是一个路标，走到终点就找到了。今天加了通配符 .，就像路标模糊了——你得把所有支路都试一遍（DFS），只要有一条路能走通就算找到。代价是通配符越多，要走的路越多。

Normal Trie: follow one road per letter. With . wildcard: the sign is blurry — you try every branch (DFS) until one succeeds. More wildcards = more roads to explore.

🗣️ 软技能 / Soft Skills — Day 77

描述你如何参与招聘和培养团队 / Describe how you've contributed to hiring and growing a team

⏱️ 阅读时间约 2 分钟 / ~2 min read | 级别 / Level: Staff

🎯 为什么面试官问这个 / Why Interviewers Ask This

Staff+ 工程师的影响力靠乘数效应——你直接写的代码是线性影响，你培养的人才是指数影响。这道题考察你是否有"建设团队"的意识，而不只是"完成任务"的执行者。

Staff+ engineers have multiplier impact — code you write is linear, engineers you grow are exponential. This question tests whether you think like a builder, not just a doer.

⭐ STAR 拆解 / STAR Breakdown

Situation（背景）:

"我们团队在做一个高并发服务重构，需要扩招两名 backend engineer，同时组内有几个经验较少的工程师需要系统化成长。"

Task（任务）:

"我负责技术面试流程的设计，同时作为两位 L4 工程师的 mentor。"

Action（行动）— 这里细节最重要:

1. 面试设计： 把 take-home project 从"写一个 REST API"改成"review 这段有问题的 PR"——更真实反映日常工作，减少刷题偏差
2. 面试一致性： 制作 rubric，让每位面试官对"Good / Strong / No hire"有统一标准
3. 培养 mentee： 不直接给答案，而是每周 1:1 问"你遇到最大的 blocker 是什么"，然后一起 root cause
4. 传递知识： 把 oncall runbook 写成 decision tree，让新人能独立处理 80% 的 alert

Result（结果）:

"6 个月内团队从 4 人扩展到 8 人，两位 mentee 都在下一个 review cycle 晋升，团队 oncall burden 降低了 40%（通过自动化 + 文档）。"

❌ Bad vs ✅ Good

👑 Senior/Staff Tips

• 量化招聘贡献： offer acceptance rate、time-to-hire、团队 ramp-up time
• 展示杠杆思维： "我花 2h 写了文档，节省了未来 6 个人各 1h"
• 避免英雄主义： 不要说"我一个人教会了整个团队"——要说你建立了系统（checklist、process、rubric）
• Staff+ 要提： 你影响的不只是直接 report，而是整个 org 的工程文化

Quantify hiring contribution. Show leverage thinking. Avoid hero narratives — show systems you built. Staff+ answer affects org-level culture, not just your immediate team.

🔑 Key Takeaways

1. 招聘不只是"找到好人"，而是设计公平、高信号的流程
2. Mentoring 的核心是问好问题，而不是给答案
3. 最强的成长证明 = mentee 晋升 或可量化的团队效率提升

📚 References

• The Manager's Path — Camille Fournier
• An Elegant Puzzle: Systems of Eng Management — Will Larson
• Staff Engineer: Leadership Beyond Management

🧒 ELI5

这道题不是问"你有没有招过人"，而是问"你有没有把团队做得更好"。最好的答案是：我不只做了一件事，我建了一套系统，让这件事以后可以更顺畅地重复发生。

This isn't "did you hire people" — it's "did you make the team better?" Best answers show: I built a system so this improvement keeps happening without me.

🤖 AI 动态 / AI News — Day 77

2026 年 6 月 AI 大事件 / June 2026 AI Highlights

⏱️ 阅读时间约 2 分钟 / ~2 min read | 模式 / Mode: NEWS

📡 本周重大动态 / This Week's Big Stories

1. 🤖 OpenAI 发布 GPT-5.6 系列（限量预览）

OpenAI Previews GPT-5.6 Family

OpenAI 开始了 GPT-5.6 系列的限量预览，包含三个档位：Sol（旗舰）、Terra（均衡）、Luna（快速低成本）。据报道，GPT-5.6 Sol 在网络安全能力上有显著增强。值得注意的是，此次发布据报道是应美国政府要求分阶段推出——OpenAI 表示这会延迟公众获取这些工具。

OpenAI began a limited preview of GPT-5.6, with three tiers: Sol (flagship), Terra (balanced), and Luna (fast/cheap). Sol reportedly features enhanced cybersecurity capabilities. The staged rollout was reportedly done at US government request — OpenAI expressed reservations about the delay to broader access.

📌 来源 / Source: OpenAI GPT-5.6 Sol preview | The Guardian

技术视角 / Tech Angle: 三档定价策略（flagship / balanced / fast）是近期行业标准，类似 Claude Opus/Sonnet/Haiku。这反映了 LLM 经济学向"按需分级"演进。

2. 🍎 苹果宣布 Siri AI — 由 Google Gemini 驱动

Apple Announces Siri AI Powered by Google Gemini

苹果宣布将语音助手品牌重塑为"Siri AI"，通过数十亿美元的合作将其接入 Google Gemini 模型。新 Siri AI 将作为独立应用运行，据报道能利用 iOS/macOS 应用上下文完成复杂任务。

Apple rebranded its voice assistant to "Siri AI," backed by a multi-billion dollar deal to power it with Google Gemini models. It runs as a dedicated app and reportedly leverages context from iOS and macOS apps for complex tasks.

📌 来源 / Source: AI News Briefs June 2026

技术视角 / Tech Angle: 苹果把 Gemini 当基础设施、而不是自研，与它一贯的垂直整合策略是重大转变。可能原因：自研大模型成本过高，时间窗口太窄。

3. 💾 OpenAI 推出自研 AI 芯片"Jalapeño"

OpenAI Reveals Custom AI Chip "Jalapeño"

OpenAI 与 Broadcom 合作推出了首款自研 AI 芯片"Jalapeño"，专为推理优化设计，目标是降低对 NVIDIA GPU 的依赖、提升推理成本效益。

OpenAI unveiled its first custom chip, "Jalapeño," built with Broadcom. Designed specifically for inference, it aims to reduce NVIDIA GPU dependence and cut per-inference costs.

📌 来源 / Source: AI key developments June 2026

技术视角 / Tech Angle: Google 有 TPU，AWS 有 Trainium/Inferentia，现在 OpenAI 也下场了。推理芯片自研=控制成本+供应链。每家顶级 AI 公司最终都会走这条路。

💡 工程师视角 / Engineer's Take

这周的主题是垂直整合：

• OpenAI 自研芯片 → 控制推理成本
• Apple 外采 Gemini → 放弃垂直整合（异类！）
• GPT-5.6 三档定价 → 市场细分成熟化

对于后端工程师，AI 推理正在成为一个基础设施问题，而不只是 ML 问题。了解 vLLM、量化、自定义芯片，已经是系统设计的核心知识。

The theme this week: vertical integration. OpenAI owns its chips, Apple outsources to Google (unusual!), and tiered pricing signals a mature market. For backend engineers: AI inference is increasingly an infra problem. vLLM, quantization, custom silicon — core systems knowledge now.

📚 References

• OpenAI GPT-5.6 Sol preview
• The Guardian — staged AI model release
• AI News June 2026 roundup

🧒 ELI5

AI 公司现在不只是在比谁的模型更聪明，还在比谁的硬件更便宜、运行更快。就像餐厅不只比菜好不好吃，还要比厨房设备是不是最省电的。OpenAI 造自己的芯片，就像米其林餐厅自己种蔬菜——控制每一个成本环节。

AI companies are now racing on hardware efficiency, not just model quality. OpenAI making its own chip is like a Michelin restaurant growing its own vegetables — controlling every cost in the chain.