系统设计：设计 A/B 测试平台 / Design an A/B Testing Platform

📅 Day 56 · 🎯 Expert Phase · ⏱️ 3 min read

🏗️ 系统设计 / System Design

想象你在设计...

你是某大型电商的平台工程师。产品经理想同时测试 10 个不同的按钮颜色，数据科学家想测试 新推荐算法，增长团队想测试 注册流程。所有实验都需要统计显著性、样本隔离、实时监控。你怎么设计这个系统？

You're a platform engineer at a large e-commerce company. Product managers want to test 10 different button colors simultaneously, data scientists want to test a new recommendation algorithm, and the growth team wants to test the registration flow. All experiments need statistical significance, sample isolation, and real-time monitoring. How do you design this system?

核心需求 / Core Requirements

功能需求 / Functional:

• 创建/管理实验 (experiment CRUD)
• 用户分桶 (user bucketing / traffic splitting)
• 事件追踪 (event tracking & metrics collection)
• 结果分析 + 统计显著性 (statistical significance)
• 实验互斥 / 互相影响检测

非功能需求 / Non-functional:

• 低延迟分桶: < 5ms p99
• 高吞吐: 1M events/sec
• 实验隔离: 修改一个不影响另一个
• 数据一致性: 同一用户永远进同一个桶

ASCII 架构图

关键设计决策 / Key Design Decisions

#### 1. 分桶策略 (Bucketing Strategy)

def assign_variant(user_id: str, experiment_key: str, variants: list) -> str:
    # Deterministic: same user always gets same variant
    # Salt per-experiment prevents correlation between experiments
    salt = f"{experiment_key}:{user_id}"
    bucket = int(hashlib.md5(salt.encode()).hexdigest(), 16) % 100
    
    # variants = [{"name": "control", "weight": 50}, {"name": "treatment", "weight": 50}]
    cumulative = 0
    for variant in variants:
        cumulative += variant["weight"]
        if bucket < cumulative:
            return variant["name"]
    return variants[-1]["name"]  # fallback

为什么用 MD5 hash？

• 均匀分布 (uniform distribution)
• 确定性：同一 userId 永远同一结果
• 快速：< 1μs

#### 2. 实验互斥 (Mutual Exclusion)

同一 Layer 内互斥；不同 Layer 可正交叠加。(Mutual exclusion within a layer; different layers are orthogonal — user can be in Exp A and Exp C simultaneously without interference.)

#### 3. 指标收集 (Metrics Collection)

Guardrail Metrics (护栏指标): 不能降低的指标 (latency, error rate, revenue)

Primary Metric: 实验的核心目标 (CTR, conversion rate)

Secondary Metrics: 了解副作用

为什么这样设计？/ Key Tradeoffs

统计方法: Frequentist (T-test) vs Bayesian — Bayesian 允许提前停止实验，但需要先验知识。大多数平台两者都提供。

别踩这个坑 / Common Mistakes

🚫 网络效应 (Network Effects): 社交产品中，用户 A 的行为会影响用户 B，违反实验独立性假设 → 解决：按 cluster/group 分桶而非个人

🚫 新奇效应 (Novelty Effect): 用户对新设计有短暂兴趣，长期效果未必好 → 解决：运行足够长时间（至少 1-2 周）

🚫 SRM (Sample Ratio Mismatch): 分配比例与实际进入比例不符，说明实验实现有 bug → 每次实验开始时做 SRM check

📚 References

• Trustworthy Online Controlled Experiments (Kohavi et al.)
• Airbnb Experimentation Platform
• Netflix Experimentation Platform

🧒 ELI5

想象你开了一家冰淇淋店，想知道巧克力口味还是草莓口味更受欢迎。你让单号桌的客人选巧克力，双号桌的客人选草莓，然后数哪边的客人吃得更开心。这就是 A/B 测试！关键是要保证每张桌子的客人是随机分配的。

Imagine you run an ice cream shop and want to know if chocolate or strawberry is more popular. You give odd-numbered tables chocolate, even-numbered tables strawberry, then count who's happier. That's A/B testing! The key is making sure customers are randomly assigned to tables.

算法：二叉树最大深度 / Maximum Depth of Binary Tree

📅 Day 56 · 💻 LeetCode #104 · 🟢 Easy · Expert Phase

💻 算法 / Algorithms

🧩 树遍历模式 (2/15) — building on the template from Day 55 (Invert Binary Tree)

今天是树遍历 Pattern Block 的第 2 题。上一题 Invert Binary Tree 教会我们：后序遍历 + 返回处理好的节点。今天我们换一种返回值语义：返回数值（深度）而不是节点。

This is problem 2/15 in the Trees pattern block. Yesterday's Invert Binary Tree taught us: post-order traversal + return processed node. Today we change the return type: return a number (depth) instead of a node.

🔗 Links

• 📝 LeetCode #104 🟢 Easy
• 📹 NeetCode Video

现实类比 / Real-World Analogy

想象你是一个侦察员，要测量一棵树的高度。你的策略：先让左右两个助手各自测量子树高度，然后取最大值 +1（加上你站的这一层）。你不需要知道整棵树的结构，只需要递归地信任你的助手。

Imagine you're a scout measuring a tree's height. Your strategy: send left and right assistants to measure sub-tree heights, then take the max + 1 (for your current level). You don't need to know the full tree structure — just recursively trust your assistants.

模版映射 / Template Mapping

# 通用模版 / Universal template:
def dfs(node):
    if not node: return BASE_CASE
    left = dfs(node.left)
    right = dfs(node.right)
    return COMBINE(node.val, left, right)

# 今天的映射 / Today's mapping:
# BASE_CASE = 0       (空节点深度为0)
# COMBINE = max(left, right) + 1   (取最大子深度 + 当前层)

Python 解法 / Solution

from typing import Optional

class TreeNode:
    def __init__(self, val=0, left=None, right=None):
        self.val = val
        self.left = left
        self.right = right

def maxDepth(root: Optional[TreeNode]) -> int:
    # Base case: empty tree has depth 0
    if not root:
        return 0
    
    # Recursively get depth of left and right subtrees
    left_depth = maxDepth(root.left)
    right_depth = maxDepth(root.right)
    
    # Current depth = max of children + 1 (for current node)
    return max(left_depth, right_depth) + 1

# Alternative: BFS (level-order) approach
from collections import deque

def maxDepthBFS(root: Optional[TreeNode]) -> int:
    if not root:
        return 0
    
    queue = deque([root])
    depth = 0
    
    while queue:
        depth += 1
        # Process entire level at once
        for _ in range(len(queue)):
            node = queue.popleft()
            if node.left:
                queue.append(node.left)
            if node.right:
                queue.append(node.right)
    
    return depth

执行追踪 / Trace

复杂度 / Complexity

DFS vs BFS 选哪个？

• 平衡树：DFS space O(log n)，BFS O(n/2) → 用 DFS
• 极度倾斜树 (skewed)：DFS space O(n)（相当于链表），BFS O(1) → 用 BFS
• 实际面试：DFS 更简洁，通常优先

🔄 Pattern 变体 / Pattern Variations

与同一 Block 的其他题对比 — 同样的模板，不同的 COMBINE 函数：

看到规律了吗？模板不变，只改 COMBINE 逻辑。 这就是模式学习的威力。

📚 References

• LeetCode #104
• NeetCode Solution
• Visualize tree recursion

🧒 ELI5

问树有多高？让左边的小助手量左边，右边的小助手量右边，然后取最高的那个 +1（加上你自己这一层）。空树高度是 0。就这么简单！

Asking how tall a tree is? Let your left helper measure the left side, right helper measure the right side, then take the taller one + 1 (for your current level). Empty tree = height 0. That's it!

软技能：快速上手陌生代码库 / Ramping Up on an Unfamiliar Codebase

📅 Day 56 · 🗣️ Soft Skills · Expert Phase · Category: Adaptability

🗣️ 软技能 / Soft Skills

面试题： 描述一次你需要快速上手陌生代码库并交付的经历。

Question: Describe a time you had to ramp up on an unfamiliar codebase quickly to deliver.

为什么这个题重要 / Why This Matters

在 Senior/Staff 级别，面试官想看的不是"你花了多少时间学习"，而是你有没有系统性地降低不确定性的方法论。这道题测试的是：

• 自主学习能力 (self-directed learning)
• 如何在缺乏上下文时优先级排序
• 如何不打扰别人的情况下快速建立心智模型

At Senior/Staff level, interviewers don't want to hear how hard you studied — they want to see a systematic methodology for reducing uncertainty. This question tests: self-directed learning, prioritization without context, and building mental models without being a burden.

STAR 拆解 / STAR Breakdown

Situation（情境）:

"我加入团队后第三周，原负责人突然离职，我需要在两周内独立完成一个关键的支付服务迁移。这个服务有 5 年历史，没有文档，原开发者已不在。"

"Three weeks after joining the team, the original owner suddenly left. I needed to independently complete a critical payment service migration in two weeks. The service was 5 years old, had no documentation, and the original developer was gone."

Task（任务）:

"在不中断线上服务的情况下，将支付处理从 Stripe v2 迁移到 v3，同时理解现有系统的所有边缘情况。"

Action（行动）:

我采用了三层理解框架：

第一层：数据流（Day 1-2） — 不看代码，先看日志和监控。用 grep 找到所有 Stripe API 调用点，画出完整的数据流图。理解"系统做什么"比"代码怎么写"更重要。

第二层：测试反向工程（Day 2-3） — 读所有现有测试，测试是最诚实的文档。从测试中提炼出隐含的业务规则。

第三层：关键路径（Day 3-5） — 找到最容易出问题的路径（失败支付、退款、webhook），重点深挖这些。其余 80% 的代码可以后补理解。

同时我建立了"未知清单"——把所有不确定的地方记录下来，按影响排序，然后集中 1 次 30 分钟和最了解系统的人对齐，而不是持续打扰。

I used a three-layer understanding framework: Layer 1: Data flow (logs & monitoring first, map all Stripe API call sites). Layer 2: Test reverse engineering (tests are the most honest documentation). Layer 3: Critical paths (focus on failure scenarios — failed payments, refunds, webhooks). I also maintained an "unknowns list" — documented all uncertainties, ranked by impact, then batched them into one 30-min sync with the most knowledgeable person.

Result（结果）:

按时完成迁移，零线上事故。迁移完成后，我写了这个服务的第一份架构文档，让下一个人不用重复我的经历。

❌ 弱回答 vs ✅ 强回答

❌ 弱回答:

"我花了很多时间读代码，遇到不懂的就问同事，慢慢就搞清楚了。"

问题：被动，没有方法论，依赖他人，面试官心里："你遇到没有同事的情况怎么办？"

✅ 强回答:

展示主动、系统性的学习框架，量化结果，以及你如何把学到的东西留下来（文档/测试/注释）让团队受益。

Senior/Staff 加分项 / Senior/Staff Tips

1. 时间盒（Timebox）你的不确定性: 不要无限深挖，设定"Day 2 结束前我必须能解释这个系统的主数据流"
2. 输出倒推输入: 先找输出（API response, 数据库写入, 消息发送），再往上追溯，比从入口向下追更高效
3. 留下痕迹: 每次理解一个模块，立刻写下来（README, ADR, test comment）。这不仅帮助自己，也建立信誉
4. 区分"需要知道"vs"好奇想知道": 在截止日期前，只深挖关键路径，其余列入后续清单

关键收获 / Key Takeaways

• 🔍 数据流优先，代码逻辑其次 — 理解系统行为比理解实现细节更快
• 📝 测试是最好的文档 — 没有文档？先读测试
• 📋 未知清单 + 批量对齐 — 减少对同事的打扰，同时确保关键问题得到解答
• 📖 学完即文档化 — 把你的理解转化为团队资产

📚 References

• The Engineering Manager's Book of Answers
• How to onboard to a legacy codebase
• Reading Code Like a Book

🧒 ELI5

就像你第一天去新学校，先找厕所在哪（最重要的！），然后找你的教室，最后才慢慢认识每个同学。先搞清楚最关键的事情，其他的慢慢来。

It's like your first day at a new school — first find the bathroom (most critical!), then find your classroom, then slowly get to know classmates. Tackle the most critical things first, everything else can wait.

AI：知识蒸馏 / Knowledge Distillation — Small Models, Big Knowledge

📅 Day 56 · 🤖 AI · Concept Mode · Expert Phase

🤖 AI — Knowledge Distillation

今日概念: 知识蒸馏 — 让小模型继承大模型的智慧

Topic: Knowledge Distillation — How Small Models Learn from Big Ones

直觉理解 / Intuitive Explanation

GPT-4 很强，但部署成本高、延迟大。能不能有一个"小 GPT-4"，既快又便宜，但效果接近？

知识蒸馏就是解法：用大模型（Teacher）来训练小模型（Student）。关键洞察：

大模型的"软标签"比真实标签包含更多信息。

比如一张猫的图片，真实标签是 [猫:1, 狗:0, 汽车:0]。但大模型输出的是 [猫:0.9, 狗:0.09, 汽车:0.01]。这个软标签告诉我们：猫更像狗，而不像汽车——这是真实标签无法传递的相对相似性信息。

GPT-4 is powerful but expensive and slow. Can we have a "small GPT-4" that's fast, cheap, but nearly as capable? Knowledge Distillation is the answer: use a large Teacher model to train a small Student model. Key insight: the Teacher's "soft labels" contain more information than ground truth labels.

工作原理 / How It Works

温度参数 T (Temperature): 控制软标签的"软度"

• T = 1: 正常 softmax
• T > 1: 分布更平滑，更多信息传递
• T < 1: 分布更尖锐，趋向 hard labels

真实案例 / Real-World Examples

可运行 Python 代码 / Runnable Code

# pip install torch transformers datasets
import torch
import torch.nn.functional as F

def distillation_loss(
    student_logits: torch.Tensor,   # Student model output
    teacher_logits: torch.Tensor,   # Teacher model output  
    true_labels: torch.Tensor,      # Ground truth
    temperature: float = 4.0,
    alpha: float = 0.7
) -> torch.Tensor:
    """
    Combined distillation loss.
    alpha controls weight: higher alpha = more reliance on teacher.
    """
    # Soft targets from teacher (with temperature scaling)
    soft_teacher = F.softmax(teacher_logits / temperature, dim=-1)
    soft_student = F.log_softmax(student_logits / temperature, dim=-1)
    
    # KL divergence loss (student learns from teacher's distribution)
    # Multiply by T^2 to preserve gradient magnitudes (Hinton et al.)
    soft_loss = F.kl_div(soft_student, soft_teacher, reduction='batchmean') * (temperature ** 2)
    
    # Hard loss (standard cross-entropy with ground truth)
    hard_loss = F.cross_entropy(student_logits, true_labels)
    
    # Combined loss
    return alpha * soft_loss + (1 - alpha) * hard_loss

# Example usage
batch_size, num_classes = 4, 10
student_logits = torch.randn(batch_size, num_classes)
teacher_logits = torch.randn(batch_size, num_classes)
labels = torch.randint(0, num_classes, (batch_size,))

loss = distillation_loss(student_logits, teacher_logits, labels)
print(f"Distillation loss: {loss.item():.4f}")
# Distillation loss: ~2.3 (varies)

蒸馏的变体 / Variants

1. Response-based Distillation（响应蒸馏）: 仅蒸馏最终输出（上述方法）
2. Feature-based Distillation（特征蒸馏）: 蒸馏中间层特征，信息更丰富
3. Relation-based Distillation（关系蒸馏）: 蒸馏样本之间的关系（相似性矩阵）
4. Chain-of-Thought Distillation（CoT蒸馏）: DeepSeek-R1 的方法——蒸馏推理链，不只是答案

局限性 / Limitations

• 🚫 容量上限: 学生模型太小时，无法吸收所有知识（压缩极限）
• 🚫 领域迁移: 教师在特定域训练，蒸馏出的学生泛化性有限
• 🚫 错误传播: 教师的偏见和错误也会被蒸馏进学生
• ⚠️ 推理能力蒸馏仍是研究热点: 复杂推理比知识更难蒸馏

📚 References

• Hinton et al. 2015 - Distilling the Knowledge in a Neural Network
• DistilBERT (Hugging Face)
• DeepSeek-R1 Distillation
• Knowledge Distillation Survey

🧒 ELI5

想象一位经验丰富的老师（大模型）和一个新学生（小模型）。学生直接背答案（真实标签）不太好，但如果老师解释"为什么猫更像狗而不像汽车"（软标签），学生就能学到更深的理解。知识蒸馏就是让小模型向大模型学习"思维过程"，而不只是"答案"。

Imagine an experienced teacher (large model) and a new student (small model). The student just memorizing answers (ground truth labels) isn't great — but if the teacher explains "why a cat looks more like a dog than a car" (soft labels), the student gains deeper understanding. Knowledge Distillation teaches small models the teacher's "reasoning process," not just the answers.