🏗️ 系统设计 / System Design — Day 29: Design a File Storage System (Google Drive/Dropbox)

难度 / Difficulty: Advanced · 阶段 / Phase: Mastery · 预计阅读时间 / Read time: ~3 min

想象这个场景 / Imagine This

你在设计 Google Drive —— 全球 10 亿用户存储文件、照片、视频，随时随地同步，多设备实时协作编辑。

You're designing Google Drive — 1B users storing files, photos, videos; syncing across devices; real-time collaborative editing.

核心挑战：任意大小的文件、版本管理、冲突解决、大规模扩展。

Core challenges: arbitrary file sizes, version control, conflict resolution, massive scale.

核心需求 / Requirements

功能性需求 / Functional:

• 上传、下载、删除文件 / Upload, download, delete files
• 文件夹层级结构 / Folder hierarchy
• 文件共享与权限管理 / Sharing & permissions
• 版本历史 / Version history (last N versions)
• 增量同步（只传改变的部分）/ Delta sync (only changed chunks)

非功能性需求 / Non-functional:

• 可用性 99.999% / 99.999% availability
• 文件一致性（跨设备）/ Cross-device consistency
• 存储 PB 级数据 / PB-scale storage

架构图 / Architecture

关键设计决策 / Key Design Decisions

1. 分块上传 / Chunked Upload

把大文件切成 4–8MB 的块（chunk），每块独立上传。

Split files into 4–8MB chunks; upload each independently.

• Why: 断点续传、重传只补缺块、去重（相同内容不重复存储）
• Why: Resume on failure, only retry failed chunks, deduplication across users

2. 内容寻址存储 / Content-Addressed Storage

块用 SHA-256 命名，相同内容 = 相同 key。

Chunks keyed by SHA-256 — identical content → same key.

• 自动去重：如果你和同事上传同一个 PDF，只存一份
• Auto-dedup: if you and a colleague upload the same PDF, stored once

3. 元数据与数据分离 / Metadata vs. Data Separation

Metadata DB stores:
  - file_id, name, parent_folder_id, owner_id
  - version_id, chunk_list (ordered SHA-256s)
  - created_at, modified_at, permissions

Chunk Store: immutable blobs in S3

4. 冲突解决 / Conflict Resolution

• 乐观锁：版本号 CAS（Compare-And-Swap）
• Optimistic locking: version CAS
• 冲突时：创建"副本（conflict copy）"，用户手动合并
• On conflict: create a "conflict copy", let user merge
• Google Docs 用 OT（Operational Transform）实现实时协作
• Google Docs uses OT for real-time co-editing

增量同步 / Delta Sync

客户端只传改变的块 → 大文件小改动 = 极少流量。

Client only sends changed chunks → small changes to big files = tiny bandwidth.

⚠️ 别踩这个坑 / Common Mistakes

容量估算 / Capacity Estimation

📚 参考资料 / References

• System Design Interview – Alex Xu: Google Drive(https://bytebytego.com/courses/system-design-interview/design-google-drive)
• Dropbox Tech Blog – How we optimized our sync algorithm(https://dropbox.tech/infrastructure/how-we-optimized-magic-pocket-for-performance)
• Google Drive API Documentation(https://developers.google.com/drive/api/guides/about-files)

🧒 ELI5

想象你要邮寄一本很厚的书给朋友。你不是把整本书一起寄，而是把每一页撕下来，分别用不同的信封寄（分块）。如果某封信丢了，只重寄那一页（断点续传）。而且如果两本书有相同的几页，只需要寄一次（去重）。

Imagine mailing a thick book to a friend. Instead of mailing the whole book at once, you tear out each page and mail them in separate envelopes (chunking). If one gets lost, you only resend that page (resume). And if two books share identical pages, you only mail them once (dedup).

💻 算法 / Algorithms — Day 30: #875 Koko Eating Bananas (Medium)

模式 / Pattern: Binary Search · 阶段 / Phase: Mastery · 预计阅读时间 / Read time: ~4 min

🧩 二分搜索模式 (3/7) — building on the template from Day 28 & 29

今天的题目是该模式的一个经典变体：搜索空间不是数组索引，而是答案本身。

Today's problem is a classic variation: the search space is the answer itself, not an array index.

📌 模式系列进度 / Pattern Progress:

1. ✅ #704 Binary Search (基础 / Basic)
2. ✅ #74 Search a 2D Matrix (2D变体 / 2D variant)
3. 👉 #875 Koko Eating Bananas (今天 / Today — 答案空间二分 / Binary search on answer)
4. ⬜ #153 Find Minimum in Rotated Sorted Array
5. ⬜ #33 Search in Rotated Sorted Array
6. ⬜ #981 Time Based Key-Value Store
7. ⬜ #4 Median of Two Sorted Arrays

题目 / Problem

🔗 LeetCode #875 – Koko Eating Bananas 🟡 Medium

📹 NeetCode Solution

题意： Koko 有 piles 堆香蕉，守卫 h 小时后回来。她每小时吃 k 根，每次只能吃一堆（吃完这堆，剩余时间不能吃别的堆）。求最小的 k 使她在 h 小时内吃完。

Koko has piles of bananas, guards return in h hours. She eats k bananas/hour, one pile at a time. Find minimum k so she finishes in h hours.

Input: piles = [3, 6, 7, 11], h = 8
Output: 4

At k=4: ceil(3/4)+ceil(6/4)+ceil(7/4)+ceil(11/4) = 1+2+2+3 = 8 hours ✓
At k=3: 1+2+3+4 = 10 hours ✗

🧩 从模版到题目 / Mapping to the Template

标准模版： 在数组上搜索 target 的位置

Standard template: Search for target position in array

今天的变体： 在 [1, max(piles)] 上搜索最小合法答案

Today's variation: Search for the minimum valid answer in range [1, max(piles)]

关键洞察 / Key Insight: 这是"在答案空间二分"的模式。当题目问"找最小的 X 使得条件成立"时，且 X 具有单调性（X越大越容易满足），就用这个模式。

This is "binary search on answer" pattern. Whenever a problem asks for "minimum X such that condition holds" and the condition is monotone (larger X = easier to satisfy), use this.

Python 解法 / Solution

from math import ceil
from typing import List

def minEatingSpeed(piles: List[int], h: int) -> int:
    # Search space: [1, max(piles)]
    # If k = max(piles), she can always finish in len(piles) <= h hours
    left, right = 1, max(piles)
    
    def can_finish(k: int) -> bool:
        # Hours needed = sum of ceil(pile / k) for each pile
        return sum(ceil(pile / k) for pile in piles) <= h
    
    result = right  # worst case: eat max pile per hour
    
    while left <= right:
        mid = (left + right) // 2
        if can_finish(mid):
            result = mid     # valid answer, try smaller
            right = mid - 1  # ← go LEFT to find minimum
        else:
            left = mid + 1   # too slow, need to go faster
    
    return result

# Trace for piles=[3,6,7,11], h=8:
# left=1, right=11, mid=6 → can_finish(6): 1+1+2+2=6 ≤ 8 ✓ → result=6, right=5
# left=1, right=5, mid=3  → can_finish(3): 1+2+3+4=10 > 8 ✗ → left=4
# left=4, right=5, mid=4  → can_finish(4): 1+2+2+3=8 ≤ 8 ✓ → result=4, right=3
# left=4 > right=3 → return 4 ✓

复杂度 / Complexity:

• Time: O(n log m) — n piles, m = max(piles)
• Space: O(1)

与模版的对比 / vs. Standard Template

举一反三 / Pattern Connections

同模式的题 / Same pattern:

• #1011 Capacity to Ship Packages — 找最小船容量，can_ship(capacity) 替代 can_finish(k)
• #410 Split Array Largest Sum — 找最小的最大子数组和
• #1482 Minimum Number of Days to Make m Bouquets — 同样是"最小X满足条件"

口诀："最小化最大值 / 最大化最小值" → 先想二分答案空间。

Mantra: "minimize the maximum / maximize the minimum" → think binary search on answer.

📚 参考资料 / References

• LeetCode #875 – Koko Eating Bananas(https://leetcode.com/problems/koko-eating-bananas/)
• NeetCode Video Explanation(https://neetcode.io/problems/eating-bananas)
• Binary Search on Answer – Patterns(https://cp-algorithms.com/binary_search.html)

🧒 ELI5

Koko 要在 8 小时内吃完所有香蕉。她应该每小时吃多少根？我们不需要一个个试 1, 2, 3...，而是用"猜数字"游戏：先猜中间值，如果太慢就往大猜，如果可以就往小猜，每次排除一半的可能性，很快就找到答案！

Koko needs to finish bananas in 8 hours. Instead of trying speeds 1, 2, 3... one by one, we play a guessing game: guess the middle speed, if too slow guess higher, if fast enough guess lower. Each guess eliminates half the possibilities — that's binary search on the answer space!

🗣️ 软技能 / Soft Skills — Day 29: How Do You Keep Your Technical Skills Current?

类别 / Category: Growth Mindset · 级别 / Level: Senior/Staff · 预计阅读时间 / Read time: ~2 min

为什么面试官问这个 / Why Interviewers Ask This

这个问题不是在问你刷了多少书。它在测试你是否有自驱的学习系统，而不是等公司培训你。Senior/Staff 工程师应该是团队里技术趋势的雷达，而不是追赶者。

This isn't about how many books you've read. It tests whether you have a self-driven learning system rather than waiting to be trained. Senior/Staff engineers should be the team's radar for tech trends, not the ones catching up.

STAR 拆解 / STAR Breakdown

Situation（背景）: 技术栈快速演进，AI、云原生、新框架不断涌现，很难跟上所有变化。

Task（任务）: 在全职工作的同时，保持技术前沿认知，并能将新知识落地到工作中。

Action（行动）—— 这是核心：

一个好的答案会提到系统化的学习策略，而不只是"我读了很多书"：

A strong answer mentions a systematic learning strategy, not just "I read a lot":

1. 过滤信号 / Signal filtering — 订阅高质量来源（TLDR Tech、Hacker News、特定作者的博客），而不是刷推特刷到焦虑

Subscribe to high-SNR sources; don't doom-scroll Twitter

1. 学了就用 / Apply immediately — 学新技术时，做一个小 side project 或在工作中找机会实践

When learning something new, build a small project or find a work opportunity to apply it

1. 输出倒逼输入 / Teach to learn — 写技术博客、在组内做分享——教别人是最好的学习方式

Blog, give team talks — teaching others forces you to understand deeply

1. 深度 vs. 广度 / Depth vs. breadth — 对核心领域深入，对周边领域保持足够认知来做决策

Go deep on your core domain; stay "enough aware" of adjacent areas to make good decisions

Result（结果）: 举一个具体例子：因为提前关注了某技术趋势，在团队做技术选型时提出了关键建议。

❌ 差的回答 vs. ✅ 好的回答

❌ Bad:

"I read tech blogs and take online courses when I have time."

太模糊，没有系统，没有影响力。

✅ Good:

"I have a weekly system: Monday I scan my RSS feeds and TLDR newsletter (30 min), flag anything relevant. Quarterly I pick one thing to go deep on — this quarter it was database internals. Last year, I noticed LLM function-calling was gaining traction, put together a proof-of-concept, and that became the foundation of our AI search feature which now serves 2M queries/day."

Senior/Staff 加分项 / Senior/Staff Level Tips

• 提到帮助团队成长，而不只是自己学习

Mention lifting the team's skills, not just your own

• 说明你如何决定什么值得深入学习（信号过滤）

Show how you decide what's worth deep-diving (signal filtering)

• 用具体数字或业务结果证明学习的价值

Use concrete outcomes to show the value of continuous learning

关键要点 / Key Takeaways

• 有系统 > 有热情：系统化学习 > 偶尔读读书
• 学了要用：能举出学以致用的具体例子
• 影响团队：Senior 级别要体现对团队的传导

System > passion: systematic learning beats occasional reading

Apply it: have concrete examples of learning → impact

Multiply impact: at Senior level, show how you elevate the team

📚 参考资料 / References

• Staff Engineer by Will Larson — Chapter: Staying Technical(https://staffeng.com/book)
• How to Stay Current as a Software Engineer – David Heinemeier Hansson(https://world.hey.com/dhh)
• The STAR Method for Behavioral Interviews – Indeed(https://www.indeed.com/career-advice/interviewing/how-to-use-the-star-interview-response-technique)

🧒 ELI5

就像医生要持续学习新的治疗方法一样，工程师也要持续学习新技术。最好的工程师不是等别人教他们，而是有自己的"学习机器"——定期看资讯、动手实践、然后教给别人。

Just like doctors must keep learning new treatments, engineers must keep learning new tech. The best engineers don't wait to be taught — they have their own "learning machine": regularly reading, building, then teaching others.

🎨 前端 / Frontend — Day 29: React Performance — Why Re-renders Happen

类别 / Category: Performance · 周 / Week: 6 · 阶段 / Phase: Mastery · 预计阅读时间 / Read time: ~2 min

真实场景 / Real Scenario

你在做一个 dashboard，用户每次点击侧边栏的某个菜单项，整个页面都会重新渲染，包括那些数据根本没变的图表组件，导致明显卡顿。你需要搞清楚 React 为什么触发了这些多余的 re-render，以及怎么阻止它。

You're building a dashboard. Every time the user clicks a sidebar item, the entire page re-renders — including chart components whose data hasn't changed — causing noticeable jank. You need to understand why React is triggering these extra re-renders and how to stop them.

React Re-render 的三个触发条件 / 3 Triggers for Re-render

⚠️ 关键误解 / Common misconception: "Props 没变就不会 re-render" — 错！ 只要父组件 re-render，子组件默认也会 re-render，哪怕 props 内容完全相同。

"If props don't change, the child won't re-render" — WRONG! By default, when a parent re-renders, ALL its children re-render too, even if props look identical.

代码示例 / Code Example

// ❌ 问题：每次 Dashboard re-render，Chart 都跟着 re-render
// ❌ Problem: Every Dashboard re-render causes Chart to re-render
function Dashboard() {
  const [selectedMenu, setSelectedMenu] = useState('overview');
  const salesData = useSalesData(); // expensive fetch

  return (
    <div>
      <Sidebar onSelect={setSelectedMenu} />
      {/* Chart re-renders even when salesData hasn't changed! */}
      <Chart data={salesData} />
    </div>
  );
}

// ✅ 解决：React.memo 跳过不必要的 re-render
// ✅ Fix: React.memo skips re-render when props are shallowly equal
const Chart = React.memo(function Chart({ data }: { data: SalesData[] }) {
  return <ExpensiveVisualization data={data} />;
});

// ⚠️ 但如果 onSelect 每次都是新函数，memo 会失效！
// ⚠️ But if onSelect is a new function reference each render, memo breaks!
function Dashboard() {
  const [selectedMenu, setSelectedMenu] = useState('overview');
  const salesData = useSalesData();

  // ✅ useCallback 稳定函数引用
  // ✅ useCallback stabilizes the function reference
  const handleSelect = useCallback((menu: string) => {
    setSelectedMenu(menu);
  }, []); // stable: no deps

  return (
    <div>
      <Sidebar onSelect={handleSelect} />
      <Chart data={salesData} /> {/* now only re-renders when salesData changes */}
    </div>
  );
}

猜猜输出什么？/ What Does This Log?

const Child = React.memo(({ onClick }: { onClick: () => void }) => {
  console.log('Child rendered');
  return <button onClick={onClick}>Click</button>;
});

function Parent() {
  const [count, setCount] = useState(0);
  const handleClick = () => console.log('clicked'); // new function each render!

  return (
    <>
      <button onClick={() => setCount(c => c + 1)}>+</button>
      <Child onClick={handleClick} />
    </>
  );
}

A) Child rendered once (on mount), never again

B) Child rendered every time you click +

C) Child rendered never (memo blocks it)

D) Error: memo requires useCallback

<details><summary>显示答案 / Show Answer</summary>

答案: B — Child rendered every time you click +

React.memo 做浅比较（shallow compare）。handleClick 每次 Parent re-render 都是新的函数引用，所以 memo 认为 props 变了，依然 re-render。

React.memo does a shallow comparison. handleClick is a new function reference on every Parent render, so memo sees "props changed" and re-renders anyway.

修复： 用 useCallback(() => console.log('clicked'), []) 稳定引用。

</details>

❌ vs. ✅ 对比 / Pattern Comparison

什么时候用 / 什么时候不用 / When to Use vs. Not

✅ 用 React.memo 当：

• 组件渲染代价高（大列表项、图表、复杂计算）
• 父组件频繁 re-render 但子组件 props 很少变

❌ 不用 React.memo 当：

• 组件很简单（div + text），memo 的检查开销比 re-render 还贵
• props 每次都变（memo 没有意义）
• 过早优化 — 先测量，再优化

Don't reach for memo by default. Profile first with React DevTools, optimize second.

📚 参考资料 / References

• React Docs – Skipping expensive recalculations with useMemo(https://react.dev/reference/react/useMemo)
• React Docs – React.memo(https://react.dev/reference/react/memo)
• Before You memo() – Dan Abramov(https://overreacted.io/before-you-memo/)

🧒 ELI5

React 就像一个老师，每次状态变了就会叫整个班的同学重新"背课文"（重新渲染）。React.memo 就是告诉 React："如果这个同学的作业没变，就不用叫他重背了。"但如果老师每次给他的"作业题目"都是新打印的一张纸（新函数引用），React 就不知道内容其实一样，还是会叫他重背。

React is like a teacher who makes the whole class recite their lessons whenever anything changes. React.memo tells React: "If this student's homework hasn't changed, skip them." But if the teacher hands them a freshly printed copy of the same homework (new function reference) each time, React can't tell it's identical and makes them recite anyway.

🤖 AI — Day 34: Context Windows — Size, Cost, and Strategies

类别 / Category: Practical · 阶段 / Phase: Mastery · 预计阅读时间 / Read time: ~2 min

什么是上下文窗口 / What Is a Context Window?

上下文窗口是 LLM 的"工作记忆"——它能同时"看到"多少 token（大概一个英文单词 ≈ 1.3 tokens）。

A context window is the LLM's "working memory" — how many tokens it can "see" at once (roughly 1.3 tokens per English word).

2023 早期 / Early 2023:   GPT-3.5 = 4K tokens  (~3,000 words)
2024:                     GPT-4o  = 128K tokens (~96,000 words)
2025:                     Claude 3 Opus/Sonnet = 200K tokens
                          Gemini 1.5 Pro = 1M tokens (~750,000 words)

1M tokens ≈ 整本《哈利·波特》全集 × 2 — 几乎可以把整个代码库喂给模型。

1M tokens ≈ 2x the entire Harry Potter series — you can feed an entire codebase to the model.

为什么"更大的上下文"不是万能药 / Why Bigger Isn't Always Better

1. 成本与速度 / Cost & Speed

输入 token 的数量直接影响推理成本（按 token 计费）：

Input token count directly drives inference cost (pay per token):

GPT-4o: ~$2.50 per 1M input tokens
Claude Sonnet: ~$3 per 1M input tokens

1M token prompt × 每天100次请求 = $250/day just for inputs

2. "迷失在中间"问题 / "Lost in the Middle" Problem

研究发现，LLM 对开头和结尾的内容注意力最高，对中间的内容容易遗漏。

Research shows LLMs pay the most attention to the beginning and end of context; content in the middle is often missed.

Attention weight distribution (approximate):
████████░░░░░░░░░░░░░░░░████████
^start high^   ^middle low^  ^end high^

3. 延迟 / Latency

即使有 1M token 的上下文，处理全部内容的延迟比处理 10K token 的上下文高几十倍。

Even with 1M token context, processing it all takes 10-50x longer than a 10K context.

实用策略 / Practical Strategies

策略 1：RAG（检索增强生成）

不把所有文档塞进上下文，而是先检索最相关的段落。

Instead of stuffing all docs into context, retrieve only the most relevant chunks.

适合 / Good for: 大型私有知识库、文档问答、企业 AI

策略 2：分块 + 重排 / Chunking + Reranking

# RAG pipeline sketch
from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=1000,
    chunk_overlap=200,  # 20% overlap prevents boundary cutoffs
)
chunks = splitter.split_text(document)

# embed chunks → store in vector DB
# at query time: retrieve top-K → rerank → pass to LLM

最佳实践：重叠率 10-20%，避免在语义边界切断内容。

Best practice: 10-20% overlap prevents cutting off context at boundaries.

策略 3：提示压缩 / Prompt Compression

用更小的模型先压缩输入，再传给大模型（LLMLingua 等技术）。

Use a smaller model to compress the input first before sending to the big model.

策略 4：放置顺序很重要 / Position Matters

由于"迷失在中间"效应，把最重要的内容放在开头或结尾：

Due to lost-in-the-middle, put the most critical content at the beginning or end:

✅ Good: [System prompt] + [Retrieved context] + [User query]
❌ Bad:  [Long doc start] + [Critical instruction buried in middle] + [Long doc end]

什么时候用长上下文 vs. RAG / Long Context vs. RAG

可运行代码 / Runnable Python

# pip install tiktoken
import tiktoken

def count_tokens(text: str, model: str = "gpt-4o") -> int:
    """Count tokens for a given text and model."""
    enc = tiktoken.encoding_for_model(model)
    return len(enc.encode(text))

def estimate_cost(text: str, price_per_million: float = 2.50) -> float:
    """Estimate input cost in USD."""
    tokens = count_tokens(text)
    return tokens / 1_000_000 * price_per_million

sample = "The quick brown fox jumps over the lazy dog. " * 1000
print(f"Tokens: {count_tokens(sample):,}")
print(f"Estimated cost: ${estimate_cost(sample):.4f}")
# Output: Tokens: 9,000  |  Estimated cost: $0.0225

📚 参考资料 / References

• Lost in the Middle: How LLMs Process Long Contexts (Stanford)(https://arxiv.org/abs/2307.03172)
• LangChain Text Splitters Docs(https://python.langchain.com/docs/modules/data_connection/document_transformers/)
• Context Length Optimization Guide 2025(https://local-ai-zone.github.io/guides/context-length-optimization-ultimate-guide-2025.html)

🧒 ELI5

上下文窗口就像你的桌子——桌子越大，能放的书越多。现在 AI 的桌子已经大到可以放整个图书馆了。但问题是：桌子越大越贵，而且东西太多你也容易忽略中间的那本书。所以聪明的做法是：先用"图书管理员"（RAG）帮你找到最相关的几本书，再放到桌子上，又快又省钱。

A context window is like your desk — the bigger it is, the more books you can put on it. Today's AI desks can fit an entire library! But bigger desks cost more, and you're likely to miss the book in the middle of a huge pile. The smart move: use a "librarian" (RAG) to fetch only the relevant books first, then put just those on your desk. Faster and cheaper.