🏗️ 系统设计 / System Design — Design a Monitoring & Alerting System (Datadog)

Day 42 · Mastery Phase · ~3 min read

想象你在设计... / Imagine You're Designing...

你在 Stripe 担任平台工程师，负责为数千个微服务搭建监控系统。每天处理数十亿个指标数据点，当支付服务延迟超过 200ms 时必须在 30 秒内触发报警。

You're a platform engineer at Stripe, building monitoring infrastructure for thousands of microservices. The system processes billions of metric data points daily and must trigger alerts within 30 seconds when payment service latency exceeds 200ms.

架构图 / Architecture Diagram

核心组件设计 / Core Component Design

指标摄入 / Metrics Ingestion

• Pull model: Prometheus 主动拉取 endpoints（适合内部服务）
• Push model: StatsD / OpenTelemetry SDK 主动推送（适合短生命周期任务）
• Hybrid: Agent (push to gateway) → Kafka → TSDB
• 数据格式：metric_name{label=value} value timestamp

时序数据库 / Time-Series Database

• 存储层次化（Tiered Storage）:

- Hot: 1-7 天，SSD，高压缩率（Gorilla 压缩）

- Warm: 7-90 天，HDD，降采样（5min 粒度）

- Cold: 90天+，S3 Parquet，1h 粒度

• Gorilla 压缩: Facebook 开源，时序数据压缩率高达 12x
• Cardinality 爆炸问题: 高基数标签（user_id）会导致数百万个时间序列

报警引擎 / Alert Engine

去重与静默 / Dedup & Silencing

• Alert grouping: 同一服务 50 个 pod 全挂 → 合并成 1 个告警
• Inhibition rules: DB 挂了 → 抑制所有依赖该 DB 的服务告警
• Maintenance windows: 计划维护期间自动静默

关键权衡 / Key Tradeoffs

为什么用 Kafka 做缓冲？/ Why Kafka as buffer?

• 削峰填谷：流量突刺时 TSDB 不被打垮
• 多消费者：流处理、存储、审计可并行消费
• 回放能力：出 bug 时可重新处理历史数据

别踩这个坑 / Common Mistakes

1. Cardinality 爆炸 — 不要把 user_id 这类高基数字段作为 label，会创建数百万时间序列
2. 告警疲劳 — 告警太多导致 on-call 忽略。解决：分级（P0/P1/P2）+ 合并 + 抑制
3. 单点 TSDB — 时序数据库本身要高可用，用集群模式（Cortex/Thanos for Prometheus）
4. 不测试告警 — 要定期"演练"：确认告警真的能触发，不是 dead link

📚 References

• Prometheus Data Model
• Facebook Gorilla: Fast, Scalable, In-Memory Time Series Database
• How Datadog Scales with Kafka

🧒 ELI5

监控系统就像医院的体检仪器。传感器（agent）不停量你的心跳（CPU）、血压（内存）、体温（延迟）。数据存进病历（TSDB）。医生提前写好"如果体温超过 38.5 度就打电话给我"（告警规则）。一旦触发，护士（通知服务）就打电话、发短信（PagerDuty/Slack）。

A monitoring system is like a hospital's diagnostic equipment. Sensors (agents) constantly measure your heart rate (CPU), blood pressure (memory), and temperature (latency). Data is stored in medical records (TSDB). Doctors pre-define "call me if temperature exceeds 38.5°C" (alert rules). When triggered, nurses (notification service) call and text (PagerDuty/Slack).

💻 算法 / Algorithms — #21 Merge Two Sorted Lists (Easy)

Day 42 · Mastery Phase · ~4 min read

🧩 链表技巧模式 (2/11) — building on the template from Day 41

今天是链表 block 的第 2 题。昨天（Reverse Linked List）我们用了迭代指针翻转；今天换一个场景：合并两个有序链表。注意模板里的快慢指针今天不需要，改用双指针并行推进。

This is the 2nd problem in the Linked List block. Yesterday (Reverse Linked List) we used iterative pointer reversal; today we shift to a different scenario: merging two sorted lists. The fast-slow pointer template isn't needed here — instead we use two-pointer parallel traversal.

题目 / Problem

🔗 LeetCode #21 🟢 Easy · 📹 NeetCode

给定两个有序链表 l1 和 l2，将它们合并成一个新的有序链表并返回。

Given two sorted linked lists l1 and l2, merge them into a single sorted linked list.

现实类比 / Real-World Analogy

两个有序的牌堆（l1 和 l2），你每次从两堆的顶部抽一张，把小的放到结果堆。最后哪堆还有剩的，整个接上去。

You have two sorted decks of cards (l1 and l2). Each round, you peek at both tops and take the smaller card. When one deck runs out, append the rest.

映射到模式 / Map to Pattern

今天不用快慢指针，而是链表的另一个核心技巧：dummy head（哨兵节点）+ 双指针合并。

为什么用 dummy head? 避免处理"结果链表第一个节点是谁"的边界情况，统一了逻辑。

Python 解法 + 逐步追踪 / Solution + Trace

class ListNode:
    def __init__(self, val=0, next=None):
        self.val = val
        self.next = next

def mergeTwoLists(l1, l2):
    dummy = ListNode(0)   # sentinel node
    cur = dummy
    
    while l1 and l2:
        if l1.val <= l2.val:
            cur.next = l1  # attach l1 node
            l1 = l1.next   # advance l1
        else:
            cur.next = l2  # attach l2 node
            l2 = l2.next   # advance l2
        cur = cur.next     # advance result pointer
    
    # attach remaining nodes (at most one list has leftovers)
    cur.next = l1 if l1 else l2
    
    return dummy.next  # skip the dummy

# Trace: l1 = 1→3→5, l2 = 2→4→6
# Step 1: 1 < 2 → attach 1,  l1=3→5,  cur→1
# Step 2: 2 < 3 → attach 2,  l2=4→6,  cur→2
# Step 3: 3 < 4 → attach 3,  l1=5,    cur→3
# Step 4: 4 < 5 → attach 4,  l2=6,    cur→4
# Step 5: 5 < 6 → attach 5,  l1=None, cur→5
# Loop ends (l1=None). cur.next = l2 (6→None)
# Result: 1→2→3→4→5→6

复杂度 / Complexity: Time O(m+n) · Space O(1)

与同 block 题目的联系 / Connection to Block

关键洞察: #143 Reorder List 和 #23 Merge K Sorted Lists 都在内部调用今天的 merge 逻辑。学好这题，后面两道 hard/medium 就降维了。

📚 References

• LeetCode #21 官方题解
• NeetCode 视频讲解
• Linked List Patterns — NeetCode

🧒 ELI5

两个装糖果的队伍，糖果按大小排好了。你站中间，每次从两队前面各拿一颗，把小的放进新盒子。哪队先没了，把另一队剩下的全倒进去。

Two queues of candies, sorted by size. You stand in the middle, compare the front of each queue each round, and take the smaller one into your new box. When one queue runs out, pour in everything left from the other.

🗣️ 软技能 / Soft Skills — How Do You Handle Scope Creep?

Day 42 · Mastery Phase · ~2 min read

为什么这道题很重要 / Why This Matters

Scope creep 是项目失败的头号杀手。面试官问这题，不是想听你怎么"拒绝需求"，而是想看你如何在交付压力、stakeholder 期望、工程质量之间找到平衡。Senior+ 工程师要展示主动识别风险、结构化沟通的能力。

Scope creep is the #1 killer of engineering projects. Interviewers aren't testing whether you refuse requests — they want to see how you balance delivery pressure, stakeholder expectations, and engineering quality. Senior+ engineers must show proactive risk identification and structured communication.

STAR 结构 / STAR Framework

Situation（情境）

"我们在做一个新的支付对账系统，计划 8 周上线。第 4 周，PM 提出增加实时欺诈检测，这会引入 3 个新的外部 API 集成。"

"We were building a new payment reconciliation system, scoped for 8 weeks. In week 4, the PM proposed adding real-time fraud detection, requiring 3 new external API integrations."

Task（任务）

识别风险、量化影响、向相关方清晰沟通，并推动达成一致决策。

Identify the risk, quantify the impact, communicate clearly to stakeholders, and drive a consensus decision.

Action（行动）

1. 立即叫停，不要悄悄吸收 — 当场说"我需要先评估影响，不能直接答应"
2. 量化影响 — 写一页纸 impact analysis：+3 周工期，+2 名工程师，延期 2 个 sprint
3. 提出选项 — 不只说"不行"，给 3 个方案：

- Option A: 推迟原计划上线，全做

- Option B: 当前版本先上线，下个 sprint 加欺诈检测

- Option C: 用第三方 SaaS（Stripe Radar）先填坑，内部方案等下季度

1. 推动决策 — 带着选项开会，让 stakeholder 做 informed decision

Result（结果）

选了 Option C：原计划如期上线，用 Stripe Radar 临时方案。下季度我们自研方案完整上线，比临时方案准确率提高了 15%。

Chose Option C: original plan launched on time with Stripe Radar as stopgap. Next quarter, the in-house solution shipped with 15% better accuracy than the temporary solution.

❌ 不好的回答 vs ✅ 好的回答

❌ "我会直接拒绝 scope 变更，告诉 PM 不行。"

→ 听起来不灵活，不懂 business context

❌ "我会默默加班把它做完。"

→ 设错预期，可能最后两头不讨好

✅ "我会先量化影响，再提出多个方案，让 stakeholder 做知情决策。"

→ 展示工程判断力 + 沟通技巧 + 主人翁意识

Senior/Staff 加分项 / Senior/Staff Signals

• 提前预防 — 在 kickoff 时就建立 change control 流程，不是等 scope creep 发生才处理
• 区分 want vs need — 追问"这个功能不上，业务目标能达成吗？"往往 30% 的 scope 是 nice-to-have
• 建立信任储蓄 — 平时交付靠谱，说"这次先砍"才有说服力
• 用数据说话 — "加 3 个 API 集成 = +3 周 = Q3 OKR miss" 比"感觉很复杂"有力得多

关键要点 / Key Takeaways

1. 量化先于拒绝 — 先评估影响，再做决定
2. 提供选项，不只说"不" — options-based communication
3. 让 stakeholder 做决策 — 你提供信息，他们对结果负责
4. 事后复盘 — scope creep 往往暴露需求分析不足，下次在 kickoff 阶段就对齐

📚 References

• Shape Up — 37signals — 如何用 appetite 控制 scope
• Staff Engineer: Leadership Beyond the Management Track — Will Larson
• The Art of Saying No Without Saying No — Marty Cagan

🧒 ELI5

你和朋友说好一起做一个 4 层蛋糕，中途朋友说"再加冰淇淋吧！"你不是直接说不行，而是说："加冰淇淋要多买材料，要多 2 小时，我们要不要换个方案：先做好蛋糕，下次单独做冰淇淋？"

You and a friend agreed to bake a 4-layer cake. Halfway through, they say "let's add ice cream too!" Instead of saying no, you say: "Adding ice cream needs more ingredients and 2 more hours — should we finish the cake first and make ice cream separately next time?"

🎨 前端 / Frontend — Next.js Data Fetching: Server Components vs Client

Day 42 · Mastery Phase · ~2 min read

真实场景 / Real Scenario

你在做一个电商 dashboard，需要展示：

1. 商品列表（从数据库 fetch，SEO 重要）
2. 购物车数量（登录用户实时状态，每次操作都变）
3. 实时库存更新（WebSocket）

问题：这三个分别用 Server Component 还是 Client Component？

You're building an e-commerce dashboard showing:

1. Product list (fetched from DB, SEO-critical)
2. Cart count (real-time per-user state, changes on every action)
3. Live inventory updates (WebSocket)

Question: Which should be Server Component vs Client Component?

核心概念 / Core Concept

代码对比 / Code Comparison

// ✅ Server Component (app/products/page.tsx)
// No 'use client' → Server Component by default
import { db } from '@/lib/db'

export default async function ProductsPage() {
  // Direct DB access — no API route needed
  const products = await db.product.findMany({ take: 20 })
  
  return (
    <div>
      {products.map(p => (
        <ProductCard key={p.id} product={p} />  // Pass to client component
      ))}
    </div>
  )
}

// ✅ Client Component (components/CartButton.tsx)
'use client'
import { useState } from 'react'

export function CartButton({ initialCount }: { initialCount: number }) {
  const [count, setCount] = useState(initialCount)
  
  return (
    <button onClick={() => setCount(c => c + 1)}>
      Cart ({count})
    </button>
  )
}

// ❌ Common mistake: marking everything 'use client'
'use client'
import { db } from '@/lib/db'  // ❌ This will FAIL — db can't run in browser
export default async function ProductsPage() { ... }

// ❌ Another mistake: fetching in useEffect when Server Component would work
'use client'
export default function Products() {
  const [products, setProducts] = useState([])
  useEffect(() => {
    fetch('/api/products').then(r => r.json()).then(setProducts)  // ❌ Extra API route, waterfall
  }, [])
}

猜猜输出 / Quiz

// app/page.tsx (Server Component)
export default async function Page() {
  console.log('where does this log?')
  return <div>Hello</div>
}

这个 console.log 输出在哪里？/ Where does this console.log appear?

A) 浏览器 DevTools Console

B) Node.js 服务器终端

C) 两者都有

D) 不会输出，Server Components 不支持 console.log

<details><summary>显示答案 / Show Answer</summary>

B) Node.js 服务器终端

Server Components 运行在服务器上，console.log 输出在你的终端（开发时）或服务器日志（生产时）。浏览器什么都看不到。

Server Components run on the server. console.log appears in your terminal (dev) or server logs (production). Nothing appears in the browser console.

</details>

决策框架 / Decision Framework

黄金法则: 把交互部分"下推"到叶子节点，让尽可能多的组件保持 Server Component。

Golden rule: Push interactivity down to leaf nodes — keep as many components as Server Components as possible.

回答之前的问题 / Answering the Original Question

1. 商品列表 → Server Component ✅（数据库直接访问，SEO 友好，零 JS）
2. 购物车数量 → Client Component（需要 useState 跟踪变化）
3. 实时库存 → Client Component（需要 WebSocket / useEffect）

📚 References

• Next.js Server Components Docs
• When to use Server vs Client Components
• React Server Components RFC

🧒 ELI5

Server Component 就像厨房里做好的菜直接端上桌——不需要顾客自己加热，已经是成品了，而且配料表（源码、数据库密码）留在厨房里。Client Component 就像自助餐台——食材在现场，顾客可以自己选、自己操作，但多了一些互动成本。

Server Component is like food prepared in the kitchen and served ready — no heating needed at the table, and the recipe (source code, DB secrets) stays in the kitchen. Client Component is like a buffet station — ingredients are on-site and guests can interact, but with more overhead.

🤖 AI — Mixture of Experts (MoE): Efficient Large Models

Day 42 · Mastery Phase · ~2 min read

直觉解释 / Intuitive Explanation

传统大模型（Dense Model）每次处理 token 时，所有参数都参与计算。GPT-4 级别的模型有数千亿参数，每个 token 都要过一遍，这非常耗算力。

MoE（混合专家模型） 的思路完全不同：把模型拆分成多个"专家"（小型 FFN 网络），每次只激活少数几个专家，其余保持沉默。结果是：参数总量大，但每次推理的计算量小。

Traditional Dense Models activate all parameters for every token. For a model with hundreds of billions of parameters, every single token triggers the full computation — extremely expensive.

MoE (Mixture of Experts) takes a different approach: split the model into multiple "experts" (small FFN sub-networks). For each token, only a few experts are activated; the rest stay silent. Result: large total parameters, but small per-token compute.

工作原理 / How It Works

Router 的工作 / What the Router Does:

• 对每个 token，输出 N 个专家的概率分布
• 选择 Top-K 个专家（通常 K=2）
• 把 token 发给这 K 个专家处理，结果加权求和

关键参数 / Key Numbers:

• Total experts: 通常 8、64 甚至数百个
• Active experts per token: 通常只有 2 个
• Sparsity: 如果有 64 个专家每次只用 2 个，激活率 = 3%

实际应用 / Real Applications

为什么 Mixtral 8x7B 很厉害？ 推理速度接近 13B 模型，但效果接近 70B 模型。鱼和熊掌兼得。

Why Mixtral 8x7B is impressive: Inference speed similar to a 13B model, but quality approaching a 70B model. Best of both worlds.

可运行的演示 / Runnable Demo

# pip install torch
import torch
import torch.nn as nn
import torch.nn.functional as F

class MoELayer(nn.Module):
    def __init__(self, input_dim=64, num_experts=4, top_k=2):
        super().__init__()
        self.top_k = top_k
        # Each "expert" is a small FFN
        self.experts = nn.ModuleList([
            nn.Linear(input_dim, input_dim) for _ in range(num_experts)
        ])
        # Router: maps input to expert scores
        self.router = nn.Linear(input_dim, num_experts)

    def forward(self, x):
        # x: (batch, input_dim)
        scores = self.router(x)                      # (batch, num_experts)
        weights, indices = torch.topk(scores, self.top_k, dim=-1)
        weights = F.softmax(weights, dim=-1)          # normalize top-k weights
        
        output = torch.zeros_like(x)
        for k in range(self.top_k):
            expert_idx = indices[:, k]               # which expert for each item
            weight = weights[:, k].unsqueeze(-1)     # scalar weight
            for i, idx in enumerate(expert_idx):
                output[i] += weight[i] * self.experts[idx](x[i])
        return output

# Demo
layer = MoELayer(input_dim=64, num_experts=4, top_k=2)
x = torch.randn(3, 64)  # batch of 3 tokens
out = layer(x)
print(f"Input: {x.shape}, Output: {out.shape}")  # both (3, 64)

MoE 的挑战 / Challenges

1. 负载不均衡（Load Imbalance） — Router 可能总选同几个专家，其他专家"失业"。解决：在 loss 里加辅助均衡项（auxiliary load balancing loss）
2. 通信开销 — 分布式训练时，token 要路由到不同 GPU 上的专家，网络通信成为瓶颈
3. 训练不稳定 — Router 学习过程中可能出现 "expert collapse"，需要精心调参

📚 References

• Mixtral of Experts Paper — Mistral AI, 2024
• Switch Transformers — Google — Fedus et al., 2021
• DeepSeek-V3 Technical Report — DeepSeek, 2024

🧒 ELI5

想象一个公司有 64 位专家员工（数学、语言、代码、逻辑……）。来了一个任务，前台（Router）看一眼，说"这道题找 2 号数学专家和 7 号代码专家来做"，其他 62 人继续喝咖啡。这样公司总能力很强，但每次开会只叫 2 个人，效率超高。

Imagine a company with 64 specialist employees (math, language, code, logic...). A task arrives and the receptionist (Router) says "get Expert #2 (math) and Expert #7 (code) for this." The other 62 keep drinking coffee. Total capacity is huge, but only 2 people are called each time — very efficient.