Tokens, Context & the Cost of a Call

⏱ ~6 min read · 🎧 5 min listen · ✎ 3 quizzes · 🧪 ~30–45 min lab

Here's a moment you will live many times as an FDE: a customer describes a feature — "summarize every support ticket as it comes in" — and looks at you. Can we do it, what will it cost, will it be fast enough? Answering that on the spot takes exactly three numbers: input tokens, output tokens, and the context window. This lesson gives you all three, then you'll make your first real API call and read them off the meter yourself.

Tokens are the meter

From Lesson 01 you know the model reads and writes tokens — chunks of text, usually part of a word. Rough rule for English prose: 1 token ≈ ¾ of a word, so 1,000 tokens is roughly 750 words.¹ Everything is denominated in tokens: pricing, speed, and limits. Words don't matter; tokens do.

And tokenization isn't uniform. Code, JSON, and rare words split into more tokens per word than plain prose — a config blob can cost triple what an equivalent-length sentence does. That's why an FDE never estimates from word counts: you count tokens (the API has a free endpoint for exactly this — it's Part 1 of your lab).

What a call costs

Pricing is per million tokens, and here's the part people miss: output tokens cost about 5× more than input tokens. Reading is cheap; generating is expensive — every output token is one pass of the next-token loop.¹ Current Claude pricing² as of mid-2026:

The context window is the model's working memory

The context window caps how much the model can consider at once — prompt plus answer. Modern Claude models take 1M tokens (~2,000 pages); Haiku takes 200K.² Two practical consequences: anything the model must "know" has to fit, and since input tokens cost money, stuffing the window is a cost decision too. "Just paste in all 10 years of customer docs" is rarely the right answer — that instinct is what leads to RAG, a few lessons from now.

Latency: the answer length is the clock

Reading input is fast and parallel; generating output happens one token at a time through the loop from Lesson 01. So response time is dominated by output length, not input length. Two levers every FDE uses: cap or prompt for shorter outputs, and stream the response so the user sees words immediately — the time to first token is what a user feels, and streaming makes a 20-second answer feel instant.

🧪 Lab: your first API call

Time to touch the metal. The lab file is in your workspace at labs/0002-first-api-call.py. One-time setup, then run it:

# one-time setup, from the repo root (PowerShell)
pip install -r requirements.txt
copy .env.example .env      # then paste your key from console.anthropic.com

python labs/0002-first-api-call.py

Three parts, each proving something from this lesson with a tight feedback loop: Part 1 counts tokens on four strings — watch JSON cost ~2× the tokens of prose. Part 2 makes a real call and reads response.usage — the exact meter you'll use to price customer features — then computes the dollar cost. Part 3 streams a response and measures time-to-first-token versus total time, so you see that generation, not reading, is the slow part. The whole lab costs under a cent.

You are a senior AI engineer reviewing my work: the output of my first LLM API lab (labs/0002-first-api-call.py) plus my scoping math. My paste includes token counts for four test strings, the response.usage numbers from a real call with my computed dollar cost, and my time-to-first-token vs. total-time measurements from a streamed call.

Grade each item as Strong / Adequate / Missing, with one sentence of evidence:
- I correctly read input vs. output tokens off the usage meter and my dollar math applies the right per-million rates to each.
- I noticed and can explain why JSON/code tokenized heavier than plain prose.
- My latency interpretation attributes total time to output generation, and I can say why TTFT matters to a user.
- My scoping estimate (cost per call and per month) is arithmetically right and states which of input or output dominates.

Be skeptical — challenge my weakest number first. Then ask me 2–3 follow-up questions a customer would ask about cost or latency. Finish with the single highest-leverage improvement.

My work follows below.

Check yourself — scope the feature

Same drill as Lesson 01: customer scenarios, and you diagnose from the mechanism. Don't scroll up. Wrong picks stay live.