Image Source: XKCD

Act 1: From Chat to Action

In Part 1: ChatGPT Doesn’t Think. It Pays Attention, we opened up the hood and looked at how LLMs actually work. From how they tokenize the input text, embed it into vectors, run it through layers of attention, and predict the next token. As well as how they continue to do so until they get a token to end/stop. That is the core of every AI tool you use today, whether it is ChatGPT, Claude, Gemini, Grok, GLM, Kimi, etc. It is attention and next-token prediction, all the way down.

But here is the thing. LLMs, on their own, can only talk. They produce text, and stop at that. They cannot read your files. They cannot run code. They cannot search the web, book a flight, send an email, or do anything in the real world. They are incredibly sophisticated text generators, but text generators nonetheless.

And yet, tools like Claude Code, Cursor, OpenAI’s Codex, and GitHub Copilot clearly do things. They, for instance, read your codebase, edit files, run tests, fix bugs, and commit changes. If LLMs can only produce text, how are these tools taking actions in the real world?

The answer is surprisingly simple, and that is what this post is about.

Image Source: Inyoung Cheong: Consumer Reports

If you have been following the AI space at all, you have probably heard the word “agent” thrown around a lot. It is one of those terms that means something different depending on who you ask. Some people use it to mean autonomous AI that can operate independently. Others use it to mean any AI that can do more than chat. The marketing around it tends to make it sound like futuristic (which it is) but also really complex.

It’s really not.

Simon Willison has, perhaps, the clearest definition I have found: an agent is an LLM that runs tools in a loop to achieve a goal. That is it. An LLM (the language brain), tools (the hands), and a loop (repeated attempts with feedback). If you understand those three things, you understand what every agentic product is doing under the hood.

In this post, we will build up from “what is a tool?” to “what is an agent?” to “how do coding agents like Claude Code actually work?” And then we will build a tiny one ourselves in about 80 lines of Python. By the end, the magic should feel a lot less magical and a lot more mechanical.

Chatbot vs. Agent: The Key Difference

Before we get into the details, it helps to draw a clear line between two things that look similar but work very differently.

A chatbot is a single exchange portal. You send a message, the LLM processes it, and it sends back a response. The LLM cannot take actions, check results, or try again if something goes wrong. This is what you are doing when you ask ChatGPT a question and get an answer. The model generates text, and that is the end of the interaction (until you type something else).

An agent is a loop. You give it a goal (“fix the failing test in auth.py”). Instead of just generating a response, it figures out what to do, takes an action (reads and/or runs the test file), observes the result (sees the error message), decides on the next step (reads the source file), takes another action (edits the code), checks the result (runs the tests), and repeats until the goal is met or it runs out of budget. This is the loop that I am concerned could replace me at my job, lol.

The difference is not the model, because the same LLM powers both. Agents, however, have a program loop wrapped around the model. So while a chatbot calls the LLM once, an agent calls the LLM in a loop, with tool execution happening between calls.

It is worth noting, though, that even familiar chatbot interfaces are starting to blur this line. For instance, if you ask ChatGPT what time it is, the model does not know the time, but it can use a tool to look it up and respond. Similarly, it can search the web to see what important events are happening in your city around that time. So while it might not be a full agent loop, the underlying mechanism is the same.

Image Source: Jakob Nielsen on UX

IBM frames agents as systems that “autonomously perform tasks by designing workflows with available tools.” GitHub describes them as AI systems that “can independently plan and execute complex tasks.” At the implementation level, it is simpler than either of those descriptions makes it sound. It is a while loop.

Here is the core of the post in one sentence: everything we are about to cover, from tool use to the agent loop to coding harnesses, is mostly just different ways of putting an LLM inside a while loop with access to functions it can call.

Let’s start with the functions.

Act 2: Tool Use

What Is a Tool?

A tool, in the context of LLMs, is just a function that the LLM can ask your program to run. It is a regular function, like one you would write in Python, JavaScript, or Bash. The only difference in this case is that you describe the function to the LLM (its name, what it does, what argument it takes, etc.), and the model gets to decide when to call it and with what input.

Let’s look at an example to see what this might look like in practice. Say you have a function called get_weather that takes a city name and returns the current weather (the details of how you get the weather don’t matter). You tell the LLM that it has access to a tool called get_weather. And that the function takes one argument, city, which is a string. The LLM can use it when someone asks about the weather. You pass this description along when you call the LLM.

Now, when a user asks “What is the weather in Seattle?” (Nobody should be asking this, though, as it should be obvious that if it’s summer, it’s sunny, if it’s not summer, it’s raining, haha), the LLM has a choice. It can either try to answer from its training data (which might be outdated or wrong), or it can output a structured tool call that says: “I want to call get_weather with city Seattle.”

Here’s an important detail. The LLM does not actually run the function. It requests that the function be run. It outputs a structured message (usually JSON) that says “call this tool with these arguments.” Then, your code, the program sitting between the user and the LLM, intercepts that request, executes the actual function, gets the result, and sends it back to the LLM. The LLM then uses the result to generate its final response.

Thus, the LLM is the decision maker. Your code, running in whatever runtime you have set up, is the executor. The LLM says what to do, your program does it, and the LLM uses your program output to improve its response.

This pattern goes by different names depending on the provider. Anthropic calls it tool use. OpenAI calls it function calling. It is the same concept, just different branding.

A Concrete Example

Let’s walk through a complete example from start to finish, so this isn’t abstract anymore.

Setup. You are building an app that lets users ask about the weather. You have access to a weather API that returns the current temperature and conditions for a given city name. You define this as a tool:

tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "input_schema": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "The name of the city"
                }
            },
            "required": ["city"]
        }
    }
]

That JSON schema describes the tool to the LLM. And you are basically telling it: this tool exists, here is what it does, here are the arguments it expects, and their types. The LLM reads this description the same way it reads any other text in its context, and it uses that understanding to decide when and how to call the tool.

Conversation. Now a user asks: “What’s the weather in Seattle?”

Here is what happens step by step:

1. User sends message. “What’s the weather in Seattle?”

2. Your code sends this to the LLM, along with the tool definitions. The LLM now knows it has a get_weather tool available.

3. The LLM responds with a tool call instead of regular text. So instead of guessing at the weather (and potentially hallucinating), it outputs something like:

{
    "type": "tool_use",
    "name": "get_weather",
    "input": { "city": "Seattle" }
}

1. Your code executes the tool. You take the city name “Seattle”, call your weather API, and get back real data:

{ "temperature": 58, "condition": "cloudy", "humidity": 72 }

1. Your code sends the result back to the LLM as a tool result message.

2. The LLM generates its final response, now grounded in real data: “It’s currently 58°F and cloudy in Seattle, with 72% humidity.”

Here is what that looks like in simplified Python using the Anthropic SDK:

import anthropic
import requests

client = anthropic.Anthropic()

# The actual function that gets weather data
def get_weather(city):
    response = requests.get(f"https://api.weather.example.com/current?city={city}")
    return response.json()  # e.g. {"temperature": 58, "condition": "cloudy", "humidity": 72}

# The tool definition: tells the LLM what this function does and what arguments it takes
tools = [
    {
        "name": "get_weather",
        "description": "Get the current weather for a city",
        "input_schema": {
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "The name of the city"
                }
            },
            "required": ["city"]
        }
    }
]

# Step 1: Send the user's message with the tools available
response = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    tools=tools,
    messages=[
      {
        "role": "user", 
        "content": "What's the weather in Seattle?"
      }
    ]
)

# Step 2: The LLM responds with a tool call
# response.stop_reason == "tool_use"
# response.content contains a tool_use block:
# {"type": "tool_use", "id": "toolu_abc123", "name": "get_weather", "input": {"city": "Seattle"}}

tool_use_block = next(b for b in response.content if b.type == "tool_use")

# Step 3: YOUR code executes the actual function
weather_data = get_weather(tool_use_block.input["city"])  # calls the real weather API

# Step 4: Send the tool result back to the LLM
follow_up = client.messages.create(
    model="claude-opus-4-7",
    max_tokens=1024,
    tools=tools,
    messages=[
        {"role": "user", "content": "What's the weather in Seattle?"},
        {"role": "assistant", "content": response.content},
        {"role": "user", "content": [
            {
                "type": "tool_result",
                "tool_use_id": tool_use_block.id,
                "content": str(weather_data)
            }
        ]}
    ]
)

# Step 5: The LLM now responds with a grounded answer
print(follow_up.content[0].text)
# "It's currently 58°F and cloudy in Seattle, with 72% humidity."

There are a few things worth noticing here.

First, the LLM never touched the weather API. It decided what to call and with what arguments, but your code did the actual execution. The LLM itself is sandboxed; so while it can request actions, it cannot take them directly. This is important for safety, as you can control what actually runs, and for flexibility (as you can connect any tool you want, from databases to APIs, file systems, and shell commands).

Second, the tool description matters. If you describe the tool poorly, the LLM will not know when to use it or how to call it correctly. If you describe it well, the LLM will use it at the right time with the right arguments. Writing good tool descriptions is a real skill, similar to writing good prompts.

Thirdly, if you look at the message flow, it is a conversation. It starts with the user message, the assistant’s response (tool call), the tool’s execution and result, and finally the assistant’s final response. Like in a regular chat, the LLM is still predicting the next token. However, in this case, one of the possible “next tokens” is a structured tool call instead of natural language text. The model learns when to emit tool calls during training, and the tool definitions in context tell it which tools are available for the current conversation.

From One Tool Call to Many

Image Source: marketoonist

Our weather example was a single tool call. And our entire flow was a simple question, with only one function available that produced a single result/answer. Real-world tasks are rarely that simple.

For instance, we can think about what happens when someone asks a coding agent to “find all the TODO comments in a project and make a checklist.” A request like this requires multiple steps. We could say:

1. List the files in the project directory

2. Read each file and search for TODO comments

3. Collect all the results

4. Format them into a checklist

Each step will have a separate tool call. The LLM could call list_files to get a list of filenames. After that, it calls read_file on each of them, reads the contents, finds and extracts the TODOs, and then synthesizes everything into a response. Each tool result becomes the context for the next decision.

Another approach could be for the LLM to call a grep tool to search for lines containing the text “TODO”, “todo”, “to do”, etc. And then it could call a read_lines tool to read the lines around the todo pattern, find details about each todo, and synthesize the output into a response.

Or let’s consider another task that we referenced earlier (a much harder one): “fix the failing test in test_auth.py.” The steps could look like:

1. Read test_auth.py to see what test is failing.

2. Run the test to see the actual error message.

3. Read the source file that the test is testing.

4. Edit the source file to fix the bug.

5. Run the test again.

6. If it fails again, read the error, edit again, run again.

7. If it passes, report success.

That is seven or more tool calls, with the LLM deciding on each step based on the results of the previous step. And at step 6, the key insight is that the LLM can fail, see the failure, and try again. This way, it is not just executing in a fixed sequence. Instead, it adapts and takes action based on feedback.

This is how tool calling becomes a loop. The LLM calls a tool, observes the result, decides what to do next, then calls another tool, observes the result, and keeps going. The conversation history grows with each step, and the LLM continues to use that full history to make each subsequent decision.

This loop is what turns an LLM from a chatbot into an agent.

Act 3: The Agent Loop

The Core Concept

At the end of Act 2, we saw how multiple tool calls naturally chain together. The LLM, as we described, calls a tool, gets a result, uses the result to decide what to do next, such as calling another tool, and so on. That chain is the agent loop, and implementing it is interestingly simple.

Here’s what it could look like in pseudocode:

while not done:
  1. Send the conversation history to the LLM
  2. The LLM responds with either:
    a. Text (it is done, or it wants to say something)
    b. A tool call (it wants to do something)
  3. If tool call:
    a. Execute the tool
    b. Add the result to the conversation
    c. Go back to step 1
  4. If text, we are done

Image Source: XKCD

Given that agent architecture, we can now do a rough implementation in Python:

import anthropic

client = anthropic.Anthropic()

def agent_loop(user_message, system_prompt, tools):
    messages = [{"role": "user", "content": user_message}]

    while True:
        response = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=4096,
            system=system_prompt,
            tools=tools,
            messages=messages
        )

        # Add the assistant's response to the conversation history
        messages.append({"role": "assistant", "content": response.content})

        # If the model is done talking, return the final text
        if response.stop_reason == "end_turn":
            return response.content[0].text

        # If the model wants to use a tool, execute it
        if response.stop_reason == "tool_use":
            tool_results = []
            for block in response.content:
                if block.type == "tool_use":
                    result = execute_tool(block.name, block.input)
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": result
                    })

            # Add tool results to the conversation and loop back
            messages.append({"role": "user", "content": tool_results})

While running, our agent loop continues until the LLM decides it is done and responds with text instead of a tool call. The underlying logic itself makes it very simple and accessible to have an agent up and running. After that, we can naturally layer additional logic and complexity, such as parallelism and multi-agent workflows, on top of the agent loop to make it more sophisticated.

Thorsten Ball, who works on Ampcode (a coding agent), in describing this logic, wrote, “It’s an LLM, a loop, and enough tokens. The rest is elbow grease.” Mihail Eric demonstrated this by building a working Claude Code clone in roughly 200 lines of Python with a loop, some tool definitions, and API calls. Similarly, the Anthropic guide on building agents describes this same pattern.

I must confess that, when I started learning about this, I was surprised to find that there is no special “agent mode” in the model. Realizing that helped me feel confident that I can utilize AI to build sophisticated workflows. And honestly, almost anybody could, too.

Why the Loop Works

You might be wondering now, if the LLM is predicting the next token, how does a simple loop produce behavior that looks so intelligent? To answer that, we can look at three properties of LLMs that enable this.

First, LLMs are good at deciding what to do next. After each tool call, the LLM receives a full conversation history that includes the original goal, every tool call it has made so far, and every result it has gotten back. Given all of that context, next-token prediction naturally tends towards the next useful action. Even though it could help, the model typically does not need a formal plan to predict what a helpful coding (or any) assistant would do next, given everything that has happened so far. And those predictions often turn out to be the right thing.

Second, LLMs can recover from errors. This is the part that makes agents feel genuinely useful. If a tool call fails (e.g., a file is not found, a test fails, a command returns an error, etc.), the error message is returned to the conversation. The LLM sees the error and can try a different approach. For instance, maybe it got the file path wrong, so it lists the directory first and shows the correct name. Maybe the code fix did not work; it can read the error message more carefully and try a different edit. This is the feedback loop in action, and it is why agents can handle tasks that are not straightforward. We can think of it as the “Think, Act, Observe” cycle: the agent allows the LLM to predict, execute, read the results, and predict again.

An interesting story that illustrates this comes from Peter Steinberger, the creator of OpenClaw. He had built a text-based travel agent to help him navigate a city, find restaurants, and translate text. He accidentally sent a voice message, even though he had never built in voice support. To his surprise, the agent responded in a few seconds. What happened was the agent recognized the audio input, searched the local machine for FFMPEG to convert the audio format, found an OpenAI API key, made a voice-to-text API call, and then responded to the transcribed message. Nobody programmed that workflow. The agent improvised it from the tools available to it. You can read about this here, or hear Peter Steinberger talk about it on TED.

Image Source: marketoonist

Third, the context window holds everything together. The entire conversation history (user messages, tool calls, tool results, the model’s own reasoning) stays in the context window. The LLM can look back at anything. It can see what the user originally asked, what it has tried, what worked, and what did not. This is why context window size matters so much for agents. A longer context window means the agent can take more steps before it runs out of space to remember what it has done. This is also directly connected to what we covered in Part 1 about attention. The model attends to the full context at every step, meaning that token 1 (the user’s original request) is just as accessible as the most recent tool result.

One more thing worth connecting back to Part 1. Remember chain-of-thought prompting and reasoning models? Those techniques make agents significantly better. When an LLM can “think” through what to do next (by generating reasoning tokens before committing to a tool call), it makes fewer mistakes and picks better actions. Extended thinking essentially gives the model scratch paper before each decision in the loop.

A Note on Context Windows

A question might be looming in your mind about what happens when we start to hit the agent’s context window limit. It is a fair question, because if the agent loop keeps appending tool calls and results to the conversation history, that history will grow with every iteration. And every iteration, the entire history gets sent back to the LLM.

Modern models have large context windows. Claude supports up to 1M tokens. Gemini also goes up to 1 million (and has experimented with even more). GPT-5.5 reaches 1,050,000 tokens. These are big numbers. To put things in perspective, 200,000 tokens is roughly 150,000 words, or about the length of two full novels. So, for most tasks, you won't run out of space.

That said, big does not mean unlimited, and there are real problems that surface well before we actually hit the limit.

The first problem is cost. LLM APIs charge per token, both input and output. Every time the agent loops, it sends the entire conversation history as input. So if your conversation is 50,000 tokens long and the agent makes 10 more tool calls, you are paying for 50,000+ input tokens on every single call. The cost grows quadratically with the task's length. Hence, long agent sessions can get expensive fast.

Image Source: marketoonist

The second problem is latency. More input tokens mean more time for the model to process the request. An agent call with 5,000 tokens of context is fast. An agent call with 100,000 tokens of context is noticeably slower. For interactive coding agents where you are watching the agent work in real time, this slowdown can become very visible.

The third problem, and honestly, the most interesting one, is attention quality. Remember from Part 1 that attention determines which information is relevant. In theory, self-attention gives every token equal access to every other token. In practice, however, research has shown that LLMs tend to pay more attention to information at the beginning and end of the context window, and less to information in the middle. This is sometimes called the “lost in the middle” problem (Liu et al., 2023). So if a critical piece of information (like the original error message that the agent is trying to fix) ends up buried in the middle of a 100,000-token conversation, the model might not attend to it as strongly as it should. The information is technically there, but the model might not make good use of it.

So how do production agents handle this? There are a few approaches.

Compaction (sometimes called summarization) is the most common technique. When the conversation gets too long, the agent pauses and asks the LLM to summarize the conversation so far into a shorter version. The summary replaces the older parts of the conversation, freeing up space while (hopefully) preserving the important details. Anthropic’s API, and generally almost all LLM providers, support this directly. But the trade-off is obvious, since summaries are lossy. The LLM has to decide what is important to keep and what to throw away, and it does not always get that right. A subtle detail from 30 tool calls ago might turn out to be exactly what the agent needs later, but if it got summarized away, it is gone.

Selective context is another approach. Instead of keeping the entire conversation history, the agent includes only the parts relevant to the current step. For a coding agent, that might mean: always include the original user request, always include the most recent few tool calls, and only include file contents when they are actively being worked on. This keeps the context focused and small, but it requires careful engineering to decide what to include and what to leave out.

Caching specifically helps with the cost problem. If the first 80% of your conversation is the same across consecutive calls (which it is, since you only append new messages at the end), you do not need to reprocess it. Techniques like prompt and KV caching (which we briefly mentioned in Part 1) allow the model to reuse computations from previous calls rather than reprocess the entire history from scratch each time.

The bottom line is that context windows are large enough for most tasks, and getting larger all the time. The real constraint, though, is not just the hard limit. It is also the cost, speed, and attention-quality trade-offs that come with filling the context window. The best agents are smart about what they put in the context and how they manage it over long sessions.

What Agents Are NOT

This is the part where I want to be honest, because there is a lot of hype around agents, and I think it is important to be clear about what they actually are right now.

Agents are not autonomous AI. The word “autonomous” is used a lot in marketing, but current agents lack goals, desires, and self-awareness. They run a loop until the loop ends. They do not decide to start working on their own. A human initiates the task, and the agent runs until it finishes, gets stuck, or hits a budget limit. That is not autonomy in any meaningful sense.

Agents are not reliable. This is the big one. Agents can and do get stuck in loops, trying the same failing approach over and over. They hallucinate tool arguments (such as calling read_file with a path that does not exist). They make confident but wrong decisions (they could edit the wrong file, or “fix” a bug by deleting the test that caught it). They sometimes do something completely unrelated to what you asked. Every person I know who uses coding agents has had stories of an agent going off the rails. They are powerful tools, but they still require supervision. The amount of supervision could be reduced over time, but we are not there yet. But who knows, DeepSeek could drop a new model tomorrow that changes everything.

Agents are not magic. This is the “emperor has no clothes” observation from Mihail Eric, and it is the central theme of this post. The core architecture of every AI agent, from the simplest scripts to products like Claude Code and Cursor, is a while loop with an LLM and some tools. The complexity in production agents is not in the architecture. It is in the engineering: error handling, safety guardrails, context management, prompt tuning, and UX polish. The core is genuinely simple.

Agents are not free. Each iteration of the loop is an LLM API call, and each one costs money. A simple task might take 3-5 tool calls. A complex task (like refactoring a module and updating all its tests) might take 30, 50, or more. The token usage adds up quickly, and this is before you count the growing context window that gets sent with every call. This is why production agents have budget limits and stop conditions. Without them, a confused agent could burn through a lot of tokens going in circles.

Types of Agents

Image Source: marketoonist

I do not want to spend too long on taxonomy because it can get academic fast, but it is helpful to know the main categories people are talking about when they say “agent.”

Retriever agents search through documents, summarize what they find, and cite sources. Think of research assistants or tools that answer questions from a company’s internal documentation. These are sometimes implemented using RAG (Retrieval-Augmented Generation).

Workflow agents perform multi-step tasks using APIs. This could include booking a flight, processing a form, ordering food, purchasing groceries, pulling data from multiple sources, and generating a report. The tools here are usually API calls to external services.

Coding agents read files, edit code, run tests, and iterate until things work. This is what Claude Code, Cursor, OpenAI’s Codex, Amp, and Aider are. We will go deep on these in Act 4.

Multi-agent systems use multiple LLMs/agents coordinating together. One agent writes code, another reviews it, a third runs tests. Or a “manager” agent delegates tasks to specialized sub-agents. These are more experimental and more complex, but the building block is still the same (an LLM in a loop with tools).

If you want the textbook taxonomy (reflex agents, goal-based agents, utility-based agents, learning agents), IBM has a good overview. Databricks also has a practical breakdown. But for the purposes of this post, we are going to focus on coding agents, because that is where the “it’s just a loop” insight is most visible and most useful.

Let’s build one.

Act 4: Coding Harnesses

What Is a Coding Harness?

Now that we understand the agent loop, let’s look at a specific, practical application: coding agents.

A coding harness (sometimes called a coding agent) is just the agent loop from Act 3 with tools designed for software development. Instead of get_weather, the tools are things like read_file, edit_file, and run_command. The LLM reads your code, decides what to change, makes the change, and checks if it worked.

This is what powers the popular coding agents like Codex, Amp, OpenCode, Cline, etc. When you use any of these tools, and it feels like the AI is “understanding” your codebase and making intelligent changes, what is actually happening is the loop we just described. The LLM reads files (tool call), thinks about what to do (next-token prediction), edits code (tool call), runs tests (tool call), sees if they pass (reads the result), and keeps going until it is done.

As we mentioned earlier, Thorsten Ball from Amp described this process as an LLM, a loop, and enough tokens. According to him, the rest is elbow grease. It turns out the “elbow grease” is quite significant (we will get to that), but the architecture underneath all of these products is the same simple loop.

The Minimal Tool Set

So what tools does our coding agent actually need? Honestly, not many. Learning that surprised me, too.

Mihail Eric built a working Claude Code clone with just three tools:

1. read_file(path) — read the contents of a file

2. list_files(directory) — list files in a directory

3. edit_file(path, old_text, new_text) — find a piece of text in a file and replace it with something else

With just these, the LLM can navigate a codebase to see what is there (list_files tool), understand the code by inspecting the files (read_file tool), and make changes to the code (edit_file tool). These three operations cover a surprisingly large range of coding tasks. In fact, if we want to push things to the extreme, we could get away with just giving the agent one tool, a bash tool that allows it access to a shell (even a pseudo/simulated shell would do), and use shell commands to see the files, read the code, and make edits. We won’t be going to that extreme in our example, but you get the idea.

Here is what the implementations of those tools could look like:

import os

def read_file(path):
    with open(path, 'r', encoding='utf-8') as f:
        return f.read()

def list_files(directory):
    entries = []
    for name in os.listdir(directory):
        full = os.path.join(directory, name)
        entries.append(f"{'[DIR] ' if os.path.isdir(full) else ''}{name}")
    return "\n".join(entries)

def edit_file(path, old_text, new_text):
    content = read_file(path)
    if old_text not in content:
        return f"Error: could not find the specified text in {path}"
    updated = content.replace(old_text, new_text, 1)
    with open(path, 'w', encoding='utf-8') as f:
        f.write(updated)
    return f"Successfully edited {path}"

Looking at how simple those functions are, it becomes immediately obvious that the intelligence is not in the tools. It is in the LLM deciding which tool to call, when to call it, and with what arguments. The tools are hands. The LLM is the brain telling the hands what to do.

Production agents add more tools on top of these. The most common additions are:

• run_command(command) — execute a shell command (run tests, install packages, compile code, check git status, etc.)

• search(pattern) — search across files using something like grep or ripgrep

• create_file(path, content) — create a new file from scratch

• delete_file(path) — remove a file

That said, the core insight that you do not need many tools still holds. As we established, three (or even one) gets you surprisingly far. Everything else is optimization.

Building a Coding Agent from Scratch

Let’s put it all together and build a minimal but functional coding agent. We need three things: a system prompt, tool definitions, and the agent loop.

The System Prompt

The system prompt tells the LLM what it is and how it should behave. For a coding agent, we want to tell it to read files before editing them and to verify its changes:

system_prompt = """You are a coding assistant. You help users with their codebase
by reading, understanding, and editing files.

Rules:
- Always read a file before editing it so you understand the current state
- After making an edit, read the file again to verify your change was applied correctly
- If something goes wrong, try a different approach
- Explain what you are doing and why"""

The Tool Definitions

These are the JSON schemas that tell the LLM what tools are available. Same format we saw in Act 2 with the weather example:

tools = [
    {
        "name": "read_file",
        "description": "Read the contents of a file at the given path",
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {
                    "type": "string",
                    "description": "The path to the file to read"
                }
            },
            "required": ["path"]
        }
    },
    {
        "name": "list_files",
        "description": "List all files and directories at the given path",
        "input_schema": {
            "type": "object",
            "properties": {
                "directory": {
                    "type": "string",
                    "description": "The directory path to list"
                }
            },
            "required": ["directory"]
        }
    },
    {
        "name": "edit_file",
        "description": "Edit a file by replacing old_text with new_text. The old_text must match exactly.",
        "input_schema": {
            "type": "object",
            "properties": {
                "path": {
                    "type": "string",
                    "description": "The path to the file to edit"
                },
                "old_text": {
                    "type": "string",
                    "description": "The exact text to find in the file"
                },
                "new_text": {
                    "type": "string",
                    "description": "The text to replace it with"
                }
            },
            "required": ["path", "old_text", "new_text"]
        }
    }
]

The Agent Loop

Now we wire it all together. This is the same loop from Act 3, but with the actual tool execution connected:

import anthropic

client = anthropic.Anthropic()

def execute_tool(name, args):
    if name == "read_file":
        return read_file(args["path"])
    elif name == "list_files":
        return list_files(args["directory"])
    elif name == "edit_file":
        return edit_file(args["path"], args["old_text"], args["new_text"])
    else:
        return f"Unknown tool: {name}"

def run_agent(user_message):
    messages = [{"role": "user", "content": user_message}]

    while True:
        response = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=4096,
            system=system_prompt,
            tools=tools,
            messages=messages
        )

        messages.append({"role": "assistant", "content": response.content})

        if response.stop_reason == "end_turn":
            # Extract and return the final text
            for block in response.content:
                if hasattr(block, "text"):
                    return block.text
            return "Done."

        if response.stop_reason == "tool_use":
            tool_results = []
            for block in response.content:
                if block.type == "tool_use":
                    print(f"  → Tool call: {block.name}({block.input})")
                    result = execute_tool(block.name, block.input)
                    print(f"  ← Result: {result[:200]}...")
                    tool_results.append({
                        "type": "tool_result",
                        "tool_use_id": block.id,
                        "content": str(result)
                    })

            messages.append({"role": "user", "content": tool_results})

# Run it
print(run_agent("Add a docstring to the main function in app.py"))

That is your entire coding agent in roughly 80 lines of Python. From here, you can embellish it, but this is roughly the starting point that Cursor, for instance, begins.

A Real Walkthrough: “Add a Docstring”

Let’s see what happens when we actually run this agent. Say you have a file called, app.py, that looks like this:

import sys

def main():
    print("Hello, world!")
    return 0

if __name__ == "__main__":
    sys.exit(main())

You ask the agent: “Add a docstring to the main function in app.py.”

Here is what happens, step by step:

Step 1. The LLM receives your message and the tool definitions. It thinks: “I need to see the current code first before I can edit it.”

Step 2. The LLM outputs a tool call: read_file(path="app.py"). Your code executes it, reads the file, and sends the contents back to the LLM.

Step 3. The LLM now sees the code. It identifies the main function and sees that it has no docstring. It decides to add one.

Step 4. The LLM outputs a tool call: edit_file(path="app.py", old_text="def main():\n print(\"Hello, world!\")", new_text="def main():\n \"\"\"Entry point for the application.\"\"\"\n print(\"Hello, world!\")"). Your code executes the find-and-replace.

Step 5. The LLM wants to verify the change, so it outputs another tool call: read_file(path="app.py"). Your code reads the file again and sends it back.

Step 6. The LLM sees the updated file with the docstring in place. Everything looks correct.

Step 7. The LLM responds with text: “Done. I added a docstring to the main function in app.py.”

A Harder Example: Fix a Failing Test

Now consider a more complex task: “Fix the failing test in test_auth.py.” (We keep coming back to this one, haha)

The trace might look something like this:

1. read_file("test_auth.py") → sees the test code, identifies the failing test

2. run_command("python -m pytest test_auth.py -x") → sees the error: AssertionError: expected 200 but got 401

3. read_file("auth.py") → reads the source code being tested

4. edit_file("auth.py", ...) → fixes the bug in the authentication logic

5. run_command("python -m pytest test_auth.py -x") → tests still fail with a different error

6. read_file("auth.py") → re-reads the file to understand the new error

7. edit_file("auth.py", ...) → tries a different fix

8. run_command("python -m pytest test_auth.py -x") → all tests pass

9. Responds: “Fixed. The issue was…”

Notice that in step 5, the fix did not work. The agent saw the failure, read the error, and tried again. This is the feedback loop that we referenced in Act 3. We see the agent adapting based on what happens at each step. This ability to recover from failure is exactly what makes agents feel very powerful.

This example also uses run_command, which our minimal three-tool agent does not have. Adding it is straightforward:

import subprocess

def run_command(command):
    result = subprocess.run(
        command, shell=True, capture_output=True, text=True, timeout=30
    )
    output = result.stdout + result.stderr
    return output[:10000]  # truncate long outputs to avoid filling up the context

With these four tools (read, list, edit, run), you can handle the majority of real coding tasks.

What Production Agents Add on Top

Our toy agent works. You could use it right now for simple tasks. But if you compare the experience to using Claude Code or Cursor, you’ll notice a significant gap. Production agents add a lot of engineering on top of the core loop. Here are some of the things that separate a weekend project like this from a real product.

Better tools. Production agents have more than our four basic tools. They include things like ripgrep for fast code search, git operations (diff, commit, checkout, etc.), LSP integration for type checking and go-to-definition, and browser tools for looking up documentation. More tools means the agent can handle more situations without getting stuck.

Image Source: marketoonist

Safety and permissions. This is critical. Our toy agent will happily run rm -rf / if the LLM decides to. Production agents have permission systems that ask for confirmation before running destructive commands, sandboxing that limits what the agent can access, allowlisted commands, and cost limits that stop the agent after a certain number of API calls. Claude Code’s permission system, for example, distinguishes between read-only operations (auto-approved) and write operations (require confirmation, unless you explicitly configure otherwise).

Context management. As we discussed in the context window section, long agent sessions fill up the context. Production agents use compaction, selective context inclusion, and caching to keep things manageable. They might summarize older parts of the conversation, only include file contents that are actively relevant, and cache previous computations to reduce cost and latency.

System prompts. Our system prompt was four lines. Production system prompts are long and carefully tuned. They include detailed instructions about coding style, error handling, when to ask for clarification versus when to proceed, safety rules, and much more. The quality of the system prompt has a huge impact on how useful the agent feels.

Streaming and UX. When you watch Codex or Cursor work, you see what the agent is doing in real time. This includes which file it is reading, what command it is running, and what changes it is making. That part of the process is not AI work. It generally takes sophisticated frontend and UX engineering to achieve, but it makes a massive difference to the user experience. Because watching an agent work in real time (and being able to interrupt it) builds trust in a way that waiting for a final answer generally does not.

Diff-based editing. Our edit_file tool uses simple string find-and-replace, which is fragile. If the LLM gets the old_text slightly wrong (for instance, an extra space, or a missing newline), the edit fails. Production agents use more robust editing strategies, such as unified diffs, AST-based edits (modifying the code’s syntax tree rather than raw text), or dedicated text editor tools designed for this purpose.

Observability. Production agents log every tool call, every tool result, every decision the LLM makes, and other relevant telemetry. When something goes wrong (and it will), you need to be able to look at the trace and understand what happened and why. Without good logging, debugging an agent would be very difficult (perhaps impossible).

In summary, while the core is simple, the product is hard. But understanding the core means you understand what every coding agent is doing (including the sophisticated ones). We have seen that the difference between our 80-line script and OpenCode is not the architecture. It is the thousands of hours of engineering, testing, and prompt tuning that make the loop reliable, safe, and pleasant to use.

If you want to go deeper, Theo’s video walks through how Claude Code works using the Thorsten Ball and Mihail Eric posts as source material. It is a great watch.

MCP: A Standard for Tools

There is one more thing worth mentioning before we close out this section.

One problem with the current landscape is that every agent has its own way of defining and connecting to tools. If you build a useful tool (say, a tool that queries your company’s database), you have to integrate it separately with every agent you want to use it with. Codex might have one tool format, and Cursor another. Your custom agent has whatever you built.

Model Context Protocol (MCP) is an open standard created by Anthropic and later donated to the Linux Foundation that aims to solve this. The idea is to build your tool server once using the MCP specification, and any MCP-compatible agent can connect to it.

The best analogy I have seen for it is USB. Before USB, every device had its own proprietary connector. Printers, keyboards, cameras, external drives, and mice all need different cables and ports. USB created one universal standard, and now everything works (mostly, especially with USB-C). MCP is trying to do the same thing for LLM tools. Build a tool once, use it everywhere.

MCP is already supported by Claude Code, Cursor, Windsurf, VS Code, OpenAI’s Codex, and others. It is still early, and the ecosystem is growing. Still, the direction is to get us to the point where, instead of every agent building its own integration from scratch, there can be a shared standard that lets tools and agents interoperate.

I am not going to go deep on MCP here because it could be its own post. But if you are building tools for LLMs, it is worth looking at the MCP documentation or this blog post to understand it and how you can use it.

Act 5: Tying It All Together

Image Source: interconnects

We have covered a lot of ground. Let’s trace the full journey one more time.

In Part 1, we learned how LLMs work. Text goes in, gets tokenized, embedded into vectors, and runs through layers of attention. The model predicts the next token, generates it, and repeats. That is the engine behind every AI chatbot you use today.

In this post, we learned how to make that engine do things. The key ideas are simple enough to fit on a napkin:

• A tool is a function that the LLM can ask your code to run.

• A tool call is the LLM saying, “run this function with these arguments” rather than generating regular text.

• An agent is an LLM in a loop, making tool calls, reading results, and deciding what to do next until the task is done.

• A coding agent is the same loop with tools for reading files, editing code, and running commands.

That is it. We built a working coding agent in about 80 lines of Python, three tools, a system prompt, and a while loop. The same architecture that sits underneath Claude Code, Cursor, Codex, and every other coding agent on the market.

We established that the “emperor has no clothes” insight from Mihail Eric is real. The core of these systems is not complicated. What is complicated is everything that makes them reliable, safe, and pleasant to use (such as prompt engineering, safety guardrails, context management, UX, error handling, and the thousands of small decisions that turn a weekend prototype into a product people trust with their codebase).

But you do not need to understand all of that to understand what is happening when you use these tools. You just need to understand the loop.

Writing this post (and Part 1) has changed how I think about the AI tools I use every day. Before I started digging into this, coding agents felt like black boxes. Now, when I watch Claude Code read a file, edit some code, run the tests, see a failure, and try again, I can see the loop. I can see the tool calls. I can see the conversation history growing. It still feels magical, but I'm confident I intuitively understand what is going on and that it is a loop interacting with a very good language model.

I think that understanding matters. Obviously, not because everyone needs to build their own agent (although you could, and maybe you should try), but because understanding what these tools are actually doing helps you use them better. You start to understand why they fail, get stuck, and sometimes do something completely unrelated to what you asked. And you start to understand what is likely to get better over time (the models, the context windows, the reasoning, the harness) and what is fundamentally a hard problem (reliability, safety, knowing when to stop).

If you want to go deeper, the resources section below has everything I found most useful while learning this material. The Thorsten Ball and Mihail Eric posts are the best starting points if you want to build your own agent. Theo’s video is a great companion if you prefer watching to reading.

Thanks for reading both parts of this series. I hope it helped demystify what is going on inside the AI tools you use every day. If you have questions, feedback, or corrections, I would love to hear from you.

AI still does not think. It still pays attention. But now it has hands.

Resources

Build Your Own Agent

• Thorsten Ball: How to Build an Agent — Build a code-editing agent in under 400 lines of Go. The best “see it with your own eyes” tutorial. “It’s an LLM, a loop, and enough tokens.”

• Mihail Eric: The Emperor Has No Clothes — Build a Claude Code clone in roughly 200 lines of Python. The post that inspired the title of this series’ central insight.

• Theo: How does Claude Code actually work? — Video walkthrough of the Thorsten Ball and Mihail Eric posts. Great companion if you prefer watching to reading.

• Anthropic Cookbook: Agent Patterns — Reference implementations for common agent workflows (prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer).

Agent Concepts and Overviews

• Simon Willison: Agents — The clearest definition of what an agent is. “An LLM agent runs tools in a loop to achieve a goal.”

• Anthropic: Building Effective Agents — Anthropic’s guide to designing agentic systems. Covers workflows vs. agents, prompt chaining, routing, parallelization, and orchestrator-worker patterns.

• Anthropic: Context Engineering for AI Agents — Deep dive into managing context in agentic systems. Covers retrieval strategies, compaction, and the shift from prompt engineering to context engineering.

• Anthropic: Writing Effective Tools for AI Agents — How to design and evaluate tools that agents can use reliably. Covers naming, descriptions, evaluation-driven development, and common failure modes.

• IBM: What are AI Agents? — Solid overview of agent concepts and how they differ from traditional automation.

• IBM: Agentic AI — Explains the broader “agentic AI” concept and how it relates to individual agents.

• IBM: Types of AI Agents — Textbook taxonomy (reflex, goal-based, utility-based, learning agents) explained clearly.

• IBM: Components of AI Agents — Breaks down what goes into an agent: perception, decision-making, action, and learning.

• GitHub: What are AI Agents? — GitHub’s overview of AI agents and how they are used in software development.

• AWS: What are AI Agents? — Amazon’s explainer covering agent architectures and evaluation.

• Google Cloud: What are AI Agents? — Google’s overview with the Think/Act/Observe framework.

• Microsoft: How do AI Agents Work? — Microsoft’s perspective on agent architecture and Copilot’s agentic features.

• MIT Sloan: Agentic AI Explained — Business-oriented explainer of agentic AI and its implications.

• Glean: 7 Core Components of AI Agent Architecture — Practical breakdown of what goes into a production agent.

• Databricks: Types of AI Agents — Pragmatic taxonomy with real-world examples.

• Evidently AI: AI Agent Examples — Collection of concrete agent use cases across industries.

• HuggingFace: Agents Course — Free course covering the Think/Act/Observe cycle, tool use, and agent design patterns.

Tool Use and Function Calling

• Anthropic: Tool Use Documentation — Official guide to implementing tool use with Claude. Covers tool definitions, message flow, and best practices.

• Anthropic: Text Editor Tool — Anthropic’s built-in text editor tool for code editing, more robust than simple find-and-replace.

• Anthropic Python SDK — Python SDK documentation with tool use examples.

• OpenAI: Function Calling Guide — OpenAI’s equivalent guide to tool use (they call it “function calling”).

Context Management

• Anthropic: Context Windows — Documentation on context window sizes and management.

• Anthropic: Compaction — How to use conversation compaction to manage long agent sessions.

• Anthropic: Prompt Caching — Reduce costs and latency by caching repeated parts of the conversation.

• Lost in the Middle: How Language Models Use Long Contexts (Liu et al., 2023) — The paper showing that LLMs attend less to information in the middle of long contexts.

• KV Caching Explained — Accessible explainer of how KV caching speeds up autoregressive generation.

MCP (Model Context Protocol)

• MCP Documentation — The official specification and getting-started guide for MCP.

• Anthropic: Introducing MCP — Anthropic’s announcement of the Model Context Protocol.

• Descope: What is MCP and How It Works — Clear third-party explainer of MCP with diagrams.

Coding Agents (Products)

• OpenAI Codex — OpenAI’s cloud-based coding agent with MCP support.

• Cursor — AI-powered code editor built on top of VS Code.

• Amp — Coding agent from the team behind Zed, built in Go.

• Aider — Open-source terminal-based coding agent for pair programming with LLMs.

• Cline — Open-source VS Code extension for autonomous coding.

• Claude Code — Anthropic’s CLI-based coding agent. Best practices guide.

• There are many more like OpenCode, Pi, RooCode, Crush, Kiro, etc.

Coding Agent Deep Dives

• Claude Code: Best Practices — Anthropic’s official guide to getting the most out of Claude Code. Covers context management, CLAUDE.md files, and effective prompting patterns.

• Anthropic: Agent Skills — How to package domain expertise into reusable skills that agents can discover and apply.

• freeCodeCamp: The Claude Code Handbook — Comprehensive guide covering installation, workflows, parallel agents, MCP integrations, and autonomous loops.

• LSP (Language Server Protocol) — The protocol that enables type checking, go-to-definition, and other IDE features that production coding agents integrate with.

• ripgrep — The fast search tool that many coding agents use under the hood for codebase search.

Image Source: XKCD

Reading guide: This is a long post (~40 min read). Here is how to navigate it.

• Want the full picture? Read straight through. Each section builds on the last.

• Already know what RNNs are? Skip to Act 2: The Transformer.

• Just want to understand GPT/ChatGPT? Jump to Act 3: From Transformer to GPT.

• Curious about scaling, RLHF, or reasoning? Head to Act 4: From GPT to LLMs.

• Short on time? Read the recap in Act 5 for the full story in a few paragraphs.

Act 1: Introduction

You’ve probably used ChatGPT today. Or maybe Google Translate, or Gemini, Claude, or one of the dozen AI tools that have quietly crept into our everyday life over the past couple of years. Typically, you type something in (or talk) and the AI just… it just responds. It feels like magic, or at least like something impossibly complex (in my opinion, it is, lol).

Personally, I started dabbling with ChatGPT back in 2023 while working on my senior thesis in college. I was building a machine learning system to detect security bugs in C/C++ code. Throughout all my prompting, I had this nagging feeling that everything I was doing was almost useless, because I knew I could give ChatGPT the code and it would find those bugs itself. Or it could be fine-tuned to do so, similar to how Wiz, the cybersecurity startup, did to a Llama model. It is such powerful technology.

The interesting thing, however, is that all these tools, every single one of them, run on the same core idea; an idea from a single paper published in 2017 by a team of researchers at Google. The paper is called “Attention Is All You Need”, and it introduced the Transformer. If you’ve heard the term and wondered what it actually means, or if you’ve never heard it and want to understand what’s going on when you talk to ChatGPT, this post is for you.

This is the first post in a two-part series. We’ll cover Transformers and LLMs here, then tackle agents and coding tools (like Claude Code) in upcoming posts. We’ll also touch briefly on the “thinking” part of LLMs (reasoning). That said, for now we’ll focus just on text, not images, audio, or other modalities, even though those are mostly built on the same underlying tech.

Let’s get into it.

The World Before Transformers

Image Source: oxen.ai

Before we can appreciate what the Transformer did, we need to understand what existed before it.

Before 2017, the dominant approach for language tasks (translation, text generation, etc.) was the Recurrent Neural Network (RNN) and its more popular variants LSTMs (Long Short-Term Memory) and GRUs (Gated Recurrent Units). The straightforward idea at the core of these models was to process text sequentially, one word at a time, from left to right, and at each step update an internal “memory” (called a hidden state) that tries to capture everything the model has read so far.

You can think of it like reading a book through a mail slot (or using one of those speed reader apps that use Rapid Serial Visual Presentation (RSVP). You see one word, you try to remember it, then the next word comes, and you try to fold that into what you already remember. However, by the time you’re 50 words in, your memory of word 1 is fuzzy at best (even if you might have a rough idea of what you read). The hidden state in RNNs is a fixed-size vector that tries to compress an ever-growing amount of information, and eventually, something has to give.

Using these models created two big problems:

1. They are slow. (Unlike speed reading with RSVP). Because RNNs process words sequentially (word 1 must be processed before word 2, word 2 before word 3, and so on), you can’t parallelize the computation. Each step depends on the previous one. This made training painfully slow, especially on longer sequences. As GPUs got faster at handling many tasks simultaneously, RNNs couldn’t take advantage of that because their architecture was fundamentally sequential.

2. They were forgetful. That fixed-size hidden state was a bottleneck. Long-range dependencies (where a word early in a sentence is crucial for understanding a word much later) are hard for the model to learn. Consider the sentence: “The cat that chased the dog that barked at the mailman sat on the mat.” By the time the model reaches “sat,” the hidden state’s memory of “cat” (the actual subject) has been diluted by everything in between. This is related to what’s known as the vanishing gradient problem: as the model learns connections over long distances, the training signal becomes weaker and weaker. Like a game of telephone, the final state has limited context on the initial states.

Researchers knew this was a problem, and they tried to fix it. LSTMs and GRUs were specifically designed to be better at retaining information across longer sequences, and they helped, but they didn’t solve the fundamental bottleneck. The hidden state remained a fixed-size summary of everything that had come before.

Then, in 2014, we had a breakthrough. Attention mechanisms (introduced by Bahdanau et al.) gave models the ability to “look back” at any part of the input when generating each word of the output, instead of relying entirely on that compressed hidden state. This was a huge improvement for tasks like translation. But attention was still being used in addition to the RNN, not instead of it. Hence, the sequential processing bottleneck still existed.

Then came the 2017 paper that changed everything. A group of researchers at Google experimented with a simple, but honestly, radical idea: what if we got rid of recurrence entirely and relied only on attention?

The result was the Transformer. And it worked so well that it became the default backbone for modern language models.

Act 2: The Transformer

Thus far, we have identified the problem: RNNs process words one at a time and struggle with long-range dependencies. We have also established that the solution is the Transformer, which relies on Attention. But what is Attention? Let’s build up our understanding using three analogies.

The Cocktail Party

Imagine you’re at a crowded party. There are dozens of conversations happening around you all at once. Music is playing, people are laughing, some people are dancing, others are drinking. Despite all the noise, when someone across the room says your name, your brain instantly locks onto that signal. You didn’t need to listen to every conversation in sequence to find the relevant one. Your brain, in a sense, just attended (or paid attention) to the right thing.

You can also split your focus. You might be talking to someone, but part of your brain is monitoring an interesting conversation happening nearby. You’re processing multiple streams of information in parallel, dynamically deciding what’s relevant and what to ignore.

That’s the core intuition behind attention in machine learning. Instead of processing information one piece at a time and hoping a hidden state remembers what matters, what if the model could “hear” the entire input at once and dynamically focus on the most relevant parts? In a Transformer, “locking on” means assigning larger numerical weights to certain tokens and smaller weights to others.

That is what the Transformer does. But “dynamically focus” is still vague, so we have to get more concrete.

The Library

Say you walk into a library with a specific research question. For instance, “What was the world like when the dinosaurs were alive?” That question is your Query or what you are looking for.

Every book on the shelf has a title and a short description on its spine. Those are the Keys, letting you know what each book is about.

Now, you scan the shelves. Your brain automatically scores how relevant each book is to your research question. A book about exactly your topic (in our case, that could be The Complete Book of Dinosaurs) would have high relevance. A book about something tangentially related, for instance, Asteroids: Astronomical and Geological Bodies, would have low relevance. And a book about something completely unrelated (let’s say a book about how Toy Story was animated) would have basically near-zero relevance.

Now, here’s where things get interesting. Instead of just grabbing the single most relevant book, you pull information from many books, each one contributing in proportion to how relevant it was to your question. These contributions are the Values, the actual content each book offers. A highly relevant book significantly enhances your understanding. A barely relevant one contributes almost nothing, but isn’t completely ignored either.

The result is a weighted combination of information from across the entire library, tailored specifically to your question. That’s attention, or a blending of all available information, weighted by relevance.

The Google Search

Now that we have the intuition from the library, let’s map it to something even more familiar and then jump straight into the math.

You type a search query into Google (or Bing — yes, people use Bing). That’s your Q. Google compares that query against the titles and descriptions of billions of pages (the Ks), scores each page’s relevance, and returns results weighted by those scores. The actual content of each page (the Vs) gets weighted by relevance, so the top results get your attention, while page-ten results barely register. One difference worth noting, though, is that Google shows you a ranked list of separate results, whereas Attention blends all the results into a single, relevance-weighted output.

The mapping here is actually clean enough that we can walk through the actual math. Here’s the equation from the paper:

If that looks intimidating, don’t worry; we’ll break it down piece by piece. But first, let’s establish some building blocks so we know what Q, K, and V actually are.

The Building Blocks

Q, K, and V are not words themselves; they’re vectors (lists of numbers) derived from the input tokens. For every token, the model creates three different vector “versions”: a Query (“what am I looking for?”), a Key (“how should others find me?”), and a Value (“what information do I contribute?”). So if your sentence has 10 tokens, you have 10 query vectors (one per token), stacked into the matrix Q. Same for K and V. Attention uses Query–Key matching to figure out what’s relevant, then uses those relevance scores to blend the Values.

But where do these vectors come from? Let’s trace the path from raw text to math.

Tokens

Models don’t operate on whole words. They operate on tokens, which are word pieces.

For example, “unbelievable” might become tokens like: [“un”, “believ”, “able”]

Each token has a unique numerical identifier.

So your sentence:

“The cat sat.”

might become tokens: [“The”, “cat”, “sat”] that map to token IDs: [713, 4583, 987]

Embeddings

Now we need to convert those IDs into a form the model can work with mathematically. That’s where embeddings come in.

An embedding is a learned lookup table in which each token ID maps to a vector (a list of numbers). At first, these vectors are random. During training, they get adjusted so that tokens used in similar contexts end up with vectors that are close together.

In practice, the Transformer has an embedding table with one row per token in its vocabulary (e.g., 50,000 rows) and a fixed number of columns (e.g., 768). Looking up an embedding is simple:

embedding_table[token_id] = vector of numbers

So if the token ID for “cat” is 4583, you look up row 4583 and get its embedding vector:

cat  ->  [0.12, -1.80, 0.03, 2.10, ...]

In real models, these vectors are typically hundreds or thousands of numbers long (e.g., 512, 768, 4096). The key idea is that similar words tend to have similar vectors. For instance, “king” and “queen” would be close together, while “king” and “toaster” would be far apart.

Position Information

There’s one more thing we need before we can feed tokens into the attention module. If we only used embeddings, these two inputs would look identical (same tokens, same embeddings):

• “cat sat on mat”

• “mat on sat cat”

Word order matters, so we add positional encoding as a way of “tagging” each token with where it appears in the sentence. In simple terms, every token’s embedding gets a small numerical adjustment based on its position. We’ll explore exactly how this works later in this post, but for now, know that after this step, each token has a vector that captures both what the token means and where it appears.

The result is that each token in your input is now represented as a vector that encodes meaning and position. These are the vectors that get transformed into Q, K, and V for the attention mechanism.

Here’s the full pipeline:

Text → Tokens → Token IDs → Embedding Vectors → Add Position → Token Vectors (ready for attention)

Now let’s break down the attention equation.

The Attention Equation, Step by Step

One important note before we begin: if your sentence has n tokens, then Q and K are each like tables with n rows (one row per token) and d_k columns. When we compute QKᵀ, the result is an n × n table of every token scored against every other token. It’s a grid of “who cares about whom.”

Step 1: Score how relevant each key is to your query.

When you Google something, the search engine needs to figure out which pages are relevant to what you typed. Mathematically, one way to measure how similar two things are is the dot product. If two vectors point in the same direction, their dot product is high; if they’re unrelated, it’s low. So we take the dot product of the Query with every Key:

Each score indicates how well a particular key matches the given query. We use the transpose of K (Kᵀ) so that the matrix dimensions line up correctly for the dot product. Q has shape (n × d_k), and Kᵀ has shape (d_k × n), giving us that (n × n) table of scores.

Step 2: Scale the scores down.

Here’s a practical problem. When the vectors are high-dimensional (as in Transformers), dot products can be very large in magnitude. Large values push the softmax function (next step) into extreme territory where one score could dominate, and everything else gets crushed to near-zero. So we divide by √d_k (the square root of the key dimension) to keep things in a reasonable range. You can think of this as volume control. We’re turning down the loudest values so the quieter ones don’t get drowned out.

Step 3: Convert scores to weights.

Right now, we have raw scores; some large, some small, some negative. We need to turn these into clean proportions that add up to 1, so we can use them as weights. That’s what the softmax function does. It takes any set of numbers and squishes them into a probability distribution: the biggest score gets the biggest weight, smaller scores get smaller weights, and everything sums to 1.

To make this concrete, let’s go back to our sentence: “The cat sat on the mat because it was tired.”

When computing attention for the token “it,” the model might assign weights like:

The model has figured out that “it” most likely refers to “cat”, so “cat” gets the lion’s share of the attention.

Step 4: Use the weights to blend the values.

Now we have a weight for every token in the input. The final step is to multiply each Value by its weight and add them all up:

The result isn’t a single vector for the whole sentence, but rather an output vector per token. Each token gets its own context-aware version of itself, updated using information from the tokens it attended to. The token “it” now carries information that’s heavily influenced by “cat,” because that’s what the attention weights determined was most relevant.

This gives us the full equation:

Hence, we have the four steps written as one expression. Score, scale, normalize, blend.

If you want to see what it looks like in code, here’s a simplified version in Python using numpy:

import numpy as np

def attention(Q, K, V):
    d_k = K.shape[-1]
    scores = Q @ K.T                # Step 1: dot product (relevance scores)
    scores = scores / np.sqrt(d_k)  # Step 2: scale down
    weights = softmax(scores)       # Step 3: convert to weights (0 to 1, sum to 1)
    output = weights @ V            # Step 4: weighted blend of values
    return output

These four steps are deceptively simple when laid out like this, but arriving at this formula took years of research, failed experiments, and the combined effort of dozens of researchers building on each other’s work. The simplicity of the final result is a testament to how much hard work it took to get there. Now let’s see what happens when you put it to work.

Self-Attention: Every Word Queries Every Other Word

Here is where things get clever. In the library analogy, we imagined one researcher walking in with one question. But in a Transformer, every word in the input is the researcher. Every single token forms its own Query and asks, “What in this sentence is relevant to me?” At the same time, every token offers up its Key and its Value to all the other tokens. And all of this happens simultaneously, in parallel, for every token in the input.

This is called self-attention (sometimes called intra-attention), and it is the mechanism that makes Transformers tick. The “self” part means that the Queries, Keys, and Values all come from the same sequence. The model is not looking at some external source of information. It is looking at itself, figuring out the internal relationships between its own tokens.

Let’s walk through what this looks like with a concrete example. Take this sentence:

“The cat sat on the mat because it was tired.”

When the model is processing the token “it,” it needs to figure out what “it” refers to. Is it the cat? The mat? Something else? With self-attention, the token “it” produces a Query vector. Every other token produces a Key vector. The model compares the Query for “it” against all of these Keys using dot products, and the score between “it” and “cat” ends up being high because the model has learned representations where these tokens are closely related in this context. The score with “mat” is lower. Similarly with “the” or “on.”

After softmax, the attention weights might look something like this:

So the output vector for “it” becomes heavily influenced by the Value vector of “cat.” In plain English, the model has updated the representation of “it” to carry information from “cat.” This resembles what linguists call coreference resolution, the problem of determining what a pronoun refers to in a text. The remarkable thing is that Transformers often learn to do this kind of behavior implicitly, just from training on large amounts of text.

And here is the part that matters most for performance. Remember how RNNs had to pass information step by step through the hidden state? If “cat” is 10 words away from “it,” the RNN needs 10 sequential steps (with 10 chances to lose information along the way) to connect them. With self-attention, the path length between any two tokens is O(1), meaning constant. It does not matter if the two words are next to each other or 500 tokens apart. The attention mechanism connects them directly in a single step. This is one of the key advantages highlighted in Table 1 of the original paper, where self-attention achieves O(1) maximum path length compared to O(n) for recurrent layers. To understand more about the notation, you can read about Big-O here.

The tradeoff is computational cost. That n × n attention matrix means self-attention scales quadratically with sequence length (O(n² · d) per layer). For short and medium sequences, this is fine, and the parallelism more than makes up for it. For very long sequences (tens of thousands of tokens), this becomes expensive, which is a big part of why running LLMs costs so much. There is a large and ongoing research effort to make attention more efficient. A good overview is this survey on efficient Transformers, and implementations like FlashAttention have made significant progress on the practical side by optimizing how attention runs on GPUs. But for the original Transformer and most models you interact with today, the full n × n attention is what runs under the hood.

One last thing before we move on. Self-attention by itself has no built-in notion of word order. The attention scores are computed purely from the content of the vectors, not from where they appear in the sentence. That is exactly why we added positional encodings back in the building blocks section. Without them, “The cat sat on the mat” and “mat the on sat cat the” would produce identical attention patterns. Positional encodings give the model the ordering information it needs so that attention can focus on what matters while still knowing where everything is.

Multi-Head Attention: More Than One Perspective

So we have self-attention. Every token can attend to every other token and produce a context-aware output. That is powerful on its own. But there is a problem.

A single attention head computes one set of attention weights. That means each token gets one “opinion” about what is relevant. But language is complicated. When you read a sentence, multiple things are going on at the same time. Consider:

“The cat sat on the mat because it was tired.”

To fully understand this sentence, you need to track several different kinds of relationships simultaneously:

• Grammar: “cat” is the subject, “sat” is the verb; they need to agree.

• Reference: “it” refers to “cat,” not “mat.”

• Proximity: “on” relates to “mat” (what it sat on) and “sat” (the action).

A single attention head must represent all these different relationships using a single attention pattern. That is a lot to ask. It is like asking one person to read a sentence, track its grammar, meaning, and structure, and report back with a single answer. The original paper describes this limitation as “averaging inhibits” the model’s ability to attend to information from different representation subspaces.

As you can tell, that would be challenging. So the solution is to ask several people rather than just one.

How It Works

Multi-head attention runs multiple attention functions in parallel, each one independently learning to focus on a different type of relationship. The original Transformer paper uses 8 heads (so 8 parallel attention functions), but other models use different numbers (12, 16, 32, etc.).

Here is how it works step by step:

Image Source: XKCD

1. Project into smaller spaces. For each head, the model takes the input vectors and projects them into a smaller set of Q, K, and V vectors using learned linear transformations (basically learned weight matrices). If the model dimension is 512 and there are 8 heads, each head works with vectors of size 512 / 8 = 64. This is where the Q, K, and V vectors actually get created. Each head learns its own set of projection weights, which means each head learns to “look at” the input differently.

2. Run attention independently. Each head takes its own Q, K, and V and runs the same attention calculation we covered earlier (dot product, scale, softmax, blend) and produces its own output.

3. Concatenate and project back. The outputs from all 8 heads are concatenated (stuck together side by side) into a single long vector. Then a final linear transformation projects it back to the original model dimension.

In equation form:

where each head is:

The Wᵢ_Q, Wᵢ_K, and Wᵢ_V are the learned projection matrices for head i. They are what allow each head to develop its own “perspective” on the input. Wᴼ is the final output projection that combines everything back together.

Here is a simplified code version (note that real implementations are more efficient, projecting once and reshaping rather than looping, but this captures the logic clearly):

def multi_head_attention(Q, K, V, W_q, W_k, W_v, W_o, num_heads):
    heads = []
    for i in range(num_heads):
        Q_i = Q @ W_q[i]           # project Q for this head
        K_i = K @ W_k[i]           # project K for this head
        V_i = V @ W_v[i]           # project V for this head
        head_i = attention(Q_i, K_i, V_i)  # run attention
        heads.append(head_i)
    
    concatenated = concat(heads)   # stick all outputs together
    output = concatenated @ W_o    # project back to model dimension
    return output

What the Heads Actually Learn

The key insight is that because each head has its own learned projections, different heads end up specializing in different things. The model is not told what to specialize in. This happens naturally during training.

The attention visualizations in the original paper (Figures 3, 4, and 5) show this clearly. For the sentence “The Law will never be perfect, but its application should be just,” different heads in layer 5 learned very different behaviors:

• One head learned to connect “its” to “Law” (resolving what “its” refers to).

• Another head learned to track the overall syntactic structure of the sentence.

• Yet another seemed to focus on adjacent word relationships.

Nobody programmed these behaviors. The heads figured them out on their own because having multiple specialized perspectives is much more effective than having one general-purpose perspective. Jay Alammar’s illustrated guide has great visualizations of this if you want to see more examples.

Why Not Just Use a Bigger Single Head?

You might wonder why we don’t just make one attention head bigger instead of using 8 smaller ones. The paper actually tested this (see Table 3, rows (A)). Single-head attention performed about 0.9 BLEU (Bilingual Evaluation Understudy), which measures the quality of generated text, points worse than the best multi-head configuration, even when the total computation was kept the same. Interestingly, too many heads also hurt performance. 8 heads turned out to be a sweet spot for their model.

The reason multi-head works better is that a single head, no matter how big, produces a single set of attention weights. It has to average across all the different types of relationships. Multi-head attention avoids this by giving each head the freedom to specialize, and then combining their outputs. It is like the difference between asking one generalist and asking a team of specialists.

A Note on Cost

You might worry that running 8 attention functions is 8 times more expensive. It is not. Because each head operates on vectors of dimension d_model / h (64 instead of 512 in the base model), the total computational cost is roughly the same as running a single attention head with the full dimension. You get the benefit of multiple perspectives without paying extra. This is pointed out in Section 3.2.2 of the original paper.

Positional Encoding: Teaching the Model Word Order

We touched on this briefly in the building blocks section, but it warrants a closer look.

Here is the problem again. Self-attention computes scores based purely on the content of the vectors. It does not care where a token appears in the sequence. Without any positional information, the model would have no way to tell apart “the cat chased the dog” and “the dog chased the cat.” They contain the same tokens, so as far as attention is concerned, they look identical. Obviously, those sentences mean very different things.

RNNs did not have this problem because they processed tokens in order. Position was baked into the architecture. But the Transformer threw out sequential processing in favor of parallelism, so it needs another way to know where things are.

The solution is to add a positional encoding to each token’s embedding before feeding it into the model. The positional encoding is a vector of the same size as the embedding, and you simply add them together element by element:

where X is the token embedding and P is the positional encoding. If your sequence has n tokens, then X is an n × d_model table of embeddings and P is an n × d_model table of positional encodings. You add them row by row, so each token’s vector ends up carrying information about both what the token is and where it appears.

How the Original Paper Does It

The authors of “Attention Is All You Need” used sine and cosine functions at different frequencies to generate positional encodings. The formulas are:

where pos is the position in the sequence (0, 1, 2, …) and i is the dimension index.

You do not need to memorize these formulas. The only thing that matters is that every position gets a unique pattern of numbers.

That said, the intuition behind them is actually straightforward. Think of it like an odometer in a car. The rightmost digit changes fastest (every mile), the next digit changes every 10 miles, the next every 100 miles, and so on. Each “digit” cycles at a different rate, so every mileage reading is unique.

Sinusoidal positional encodings work the same way. Each dimension of the encoding oscillates at a different frequency. The early dimensions cycle quickly (changing significantly between adjacent positions), while the later dimensions cycle slowly (changing gradually over long stretches of the sequence). The combination of all these frequencies at different rates gives every position a unique “fingerprint.”

This design has a nice property: for any fixed k distance between two positions, the positional encoding of pos + k can be expressed as a linear function of the encoding at pos. This means the model can potentially learn to attend by relative position (how far apart two tokens are) rather than just absolute position (what position number each one has).

Do You Have to Use Sinusoids?

No. The paper also experimented with learned positional embeddings (where the model learns a separate vector for each position during training, similar to how it learns token embeddings) and found that both approaches produced nearly identical results (see Table 3, row (E)).

The authors chose sinusoidal encodings because they hypothesized that it would allow the model to generalize to sequence lengths longer than anything it saw during training. Since the sine and cosine functions are defined for any position, the model does not encounter the “I’ve never seen position 5001” problem that a learned embedding table would.

In practice, many modern Transformer models use entirely different approaches. For instance, Rotary Position Embeddings (RoPE) have become popular in recent LLMs because they encode relative position directly into the attention computation. But the core idea remains the same: the model needs some way to know where tokens are, and positional encodings provide that.

Putting It All Together: The Full Transformer Architecture

We now have all the pieces. Let’s see how they fit together.

The Transformer follows an encoder-decoder structure, which was common in sequence-to-sequence models before it. The encoder reads the input (for example, a French sentence) and builds a rich representation of it. The decoder takes that representation and generates the output (the English translation) one token at a time.

What makes the Transformer different is how the encoder and decoder are built. Instead of recurrent layers, they are built entirely from the components we’re learning about in this post.

Let’s walk through each part.

The Encoder

The encoder is a stack of N = 6 identical layers (in the base model). Each layer has two sub-layers:

1. Multi-head self-attention. Every token in the input attends to every other token, exactly as we described. This is where the model builds context-aware representations.

2. A position-wise feed-forward network. After attention gathers information from across the sequence, this network processes each token independently. It consists of two linear transformations with a ReLU activation in between:

You can think of the division of labor this way: attention figures out what information to gather from other tokens, and the feed-forward network figures out what to do with that gathered information. The feed-forward network applies the same transformation to each token position independently, but it uses different learned parameters at each layer. In the base model, the input and output dimensions are d_model = 512, and the inner layer has a dimension of d_ff = 2048.

Each of these two sub-layers also has two additional components wrapped around it:

Residual connections. The output of each sub-layer is added back to its input before moving on. In other words, the model computes x + SubLayer(x) rather than just SubLayer(x). This idea comes from deep residual networks (ResNets), and it solves a practical problem: in deep networks (with many layers), information can get lost or distorted as it passes through layer after layer. By adding the input back to the output, the model always has access to the original information. If a layer does not learn anything useful, the residual connection lets the signal pass through unchanged rather than getting corrupted. Think of it like keeping a photocopy of a document before editing it. If the edits make things worse, you still have the original.

Layer normalization. After each residual connection, the model normalizes the values to keep them in a stable range. Without normalization, the numbers flowing through the network can gradually grow very large or very small as they pass through layer after layer, making training unstable. Layer norm keeps things well-behaved.

So the full operation for each sub-layer is:

Stack 6 of these layers on top of each other, and that is the encoder. The input tokens go in at the bottom (after embedding and positional encoding), flow through all 6 layers, and what comes out at the top is a set of context-aware representations, one per token, that capture the meaning of the entire input sequence.

The Decoder

The decoder is also a stack of N = 6 identical layers, but each layer has three sub-layers instead of two:

1. Masked multi-head self-attention. This is the same self-attention as in the encoder, with one crucial difference: masking. When the model is generating output, it produces tokens one at a time, left to right. When predicting token 5, it should only see tokens 1 through 4. It must not be able to peek at tokens 6, 7, 8, and beyond, because those have not been generated yet. The mask enforces this by setting all “illegal” attention scores (positions to the right of the current position) to a very large negative number (conceptually −∞) before the softmax step. Since softmax of a very large negative number is essentially 0, those future positions get zero attention weight, effectively making them invisible. This is what makes the model autoregressive, meaning it generates one token at a time based on what came before.

2. Cross-attention (encoder-decoder attention). This is the layer that connects the decoder to the encoder. The Queries come from the decoder (from the previous decoder layer’s output), but the Keys and Values come from the encoder’s output. This allows every decoder position to attend to all positions in the input sequence. It is how the decoder “reads” the original input. For example, when translating a French sentence into English, the decoder looks back at the French to decide which English word to produce next. This mimics the traditional encoder-decoder attention from older sequence-to-sequence models, but uses multi-head attention instead of simpler mechanisms.

3. A position-wise feed-forward network. Same as in the encoder.

Each sub-layer again has residual connections and layer normalization wrapped around it, just like in the encoder.

The Three Types of Attention

It is worth pausing to note that the Transformer uses multi-head attention in three distinct ways:

1. Encoder self-attention. Every token in the input attends to every other input token. No masking. This builds a rich representation of the input.

2. Decoder masked self-attention. Every token in the output attends to previous output tokens only. Masking prevents looking ahead. This maintains the autoregressive property.

3. Cross-attention. The decoder attends to the encoder’s output. Queries from the decoder, Keys, and Values from the encoder. This is the bridge between input and output.

These three uses of the same mechanism are one of the elegant aspects of the Transformer design. The same multi-head attention operation, applied in different configurations, handles input understanding, output generation, and input-output connection.

Embeddings and the Final Output

At the very bottom of both the encoder and decoder, input tokens are converted to vectors through the embedding layer and combined with positional encodings, as we covered in the building blocks section.

At the very top of the decoder, the output vectors need to be converted back into actual words (or more precisely, token probabilities). This is done with a linear layer followed by a softmax function:

This produces a probability distribution over the entire vocabulary. The model picks the token with the highest probability (or samples from the distribution) as its prediction.

One interesting detail from the paper: the same weight matrix is shared between the two embedding layers (encoder input and decoder input) and the pre-softmax linear transformation. This weight tying reduces the total number of parameters and was found to work well in practice.

Seeing It All Together

If you want to see this full architecture in action, the Transformer Explainer from Georgia Tech’s Polo Club is an excellent interactive tool. You can type in different inputs and watch how tokens flow through embeddings, attention layers, and feed-forward networks in real time. I highly recommend spending some time with it.

The full architecture from the original paper looks like this:

To summarize the data flow:

1. Input tokens get embedded and combined with positional encodings.

2. They pass through 6 encoder layers (self-attention + feed-forward, with residual connections and layer norm at each step).

3. The encoder’s final output is sent to the decoder.

4. The decoder takes the previously generated output tokens (also embedded with positional encodings), passes them through 6 decoder layers (masked self-attention + cross-attention + feed-forward).

5. The decoder’s output goes through a linear layer and softmax to produce the next token prediction.

6. That predicted token gets fed back in as input, and steps 4 and 5 repeat until the sequence is complete.

And that is the Transformer. Every component we covered in this section (attention, multi-head attention, positional encoding, feed-forward networks, residual connections, layer normalization) works together to create a model that can process entire sequences in parallel and learn complex relationships between tokens at any distance.

The original “Attention Is All You Need” paper tested this architecture on machine translation and achieved state-of-the-art results on both English-to-German and English-to-French benchmarks, while training significantly faster than the RNN-based models it replaced. The big Transformer model achieved a BLEU score of 28.4 on English-to-German (see Table 2), improving over the previous best by more than 2 points, after training for just 3.5 days on 8 GPUs.

But the real impact of the Transformer turned out to be far greater than that of machine translation. Researchers soon realized that parts of this architecture could be repurposed for a much more ambitious goal: building a general-purpose language model. That is where GPT comes in.

Act 3: From Transformer to GPT

The Big Bet: Decoder Only

The Transformer we just walked through was designed for translation. The encoder reads a sentence in one language, and the decoder writes it in another. It is a sequence-to-sequence machine, and it was very good at that job.

But in 2018, researchers at OpenAI made a surprising and bold bet: what if you threw away the encoder entirely and just kept the decoder?

This bet gave birth to GPT (Generative Pre-trained Transformer). It doesn’t have an encoder, and there is no cross-attention, only a stack of decoder layers with masked self-attention, trained only to predict the next token.

It takes a sequence of tokens, and the model tries to predict the next token. Hence, the training objective is: given everything that came before, what is the most likely next token?

However, to predict the next word well, the model has to implicitly learn an enormous amount about language. It has to learn grammar (so sentences are well-formed), facts (so statements are plausible), reasoning patterns (so arguments follow logically), style and tone (so text sounds natural), and much more. All of these things are baked into the patterns of which words tend to follow which other words in real text.

The training process works like this:

1. Take a large chunk of text (from the internet, books, articles, code, etc.).

2. For each position in the text, the model sees all the tokens before that position and tries to predict the token at that position.

3. Compare the model’s prediction to the actual token. Compute a loss (how wrong was it?).

4. Use backpropagation to adjust the model’s weights slightly in the direction that would have made the prediction better.

5. Repeat this billions of times, across billions of tokens.

This is called autoregressive language modeling. “Autoregressive” just means each prediction depends on the previous ones. The model generates one token, feeds it back in as input, generates the next token, feeds that back in, and so on.

The architectural changes from the full Transformer are minimal:

• No encoder. There is no separate “input understanding” module. The decoder is the whole model.

• No cross-attention. Since there is no encoder, the cross-attention layer (where the decoder would read the encoder’s output) is removed. Each decoder layer now has just two sub-layers: masked self-attention and a feed-forward network.

• Masked self-attention only. The model can never look at future tokens. When predicting token 5, it can only see tokens 1 through 4. This is the same masking we described in the decoder section, and it is essential for autoregressive generation.

That is the entire GPT architecture. The original GPT paper (Radford et al., 2018) used 12 decoder layers with a model dimension of 768, trained on BooksCorpus (roughly 7,000 books). It was solid proof of concept. What came next was much bigger.

How GPT Actually Generates Text

Understanding how GPT generates text is important because this is what is happening every time you use ChatGPT, Claude, Gemini, Grok, or any other AI chat tool. It all comes down to a loop.

Let’s walk through a concrete example. You type:

“The capital of France is”

Here is what happens:

Step 1. The model tokenizes your input: [“The”, “capital”, “of”, “France”, “is”]. Each token gets embedded, positional encodings are added, and the resulting vectors flow through all the decoder layers (masked self-attention, feed-forward, residual connections, layer norm, just like we covered).

Step 2. At the output, the model produces a probability distribution over its entire vocabulary (which could be 50,000+ tokens). Each token in the vocabulary gets a probability. “Paris” might get 0.92. “London” might get 0.03. “Berlin” might get 0.01. And so on.

Step 3. The model selects a token from this distribution. In the simplest case, it picks the one with the highest probability: “Paris.”

Step 4. That token gets appended to the input. Now the sequence is [“The”, “capital”, “of”, “France”, “is”, “Paris”]. The whole process conceptually repeats from Step 1 with this new, longer sequence (in practice, real systems use a technique called KV caching to reuse computations from previous steps and avoid redundant work). The model predicts the next token, maybe “.” or “,” and so on.

Step 5. This loop continues until the model produces a special end-of-sequence token, or until it hits a maximum length limit.

Here is a simplified version in code:

def generate(model, prompt_tokens, max_new_tokens):
    tokens = prompt_tokens
    for _ in range(max_new_tokens):
        probabilities = model(tokens)          # forward pass through all layers
        next_token = sample(probabilities)      # pick a token
        tokens = tokens + [next_token]          # append to sequence
        if next_token == END_OF_SEQUENCE:
            break
    return tokens

If you want to see a real (but still readable) implementation of this, Karpathy’s minGPT is about 300 lines of core code. It is a great way to see how simple the generation loop really is.

Temperature: Controlling Creativity

You might have noticed I said the model “selects a token from this distribution.” How it selects matters a lot, and this is controlled by a parameter called temperature.

Temperature = 0 (or very close to 0). The model always picks the single most likely token. This is deterministic so that you will get the same output every time for the same input. It tends to be more consistent, though not necessarily more correct, and can be repetitive.

Temperature = 1. The model samples proportionally to the probabilities. If “Paris” has a 0.92 probability and “Lyon” has a 0.02 probability, there is a small but real chance it picks “Lyon.” This adds variety and creativity, but it can also lead to incoherent or surprising outputs.

Temperature between 0 and 1. This is the sweet spot for most applications. The distribution is “sharpened” so high-probability tokens are even more favored, but there is still some randomness. Most AI tools you use operate somewhere in this range.

This is why you can ask ChatGPT the same question twice and get different answers. It is not “thinking differently” each time. The underlying probabilities remain the same, but random sampling selects different paths through the distribution.

Pre-training: Learning from the Internet

So, where does GPT get its knowledge? Pre-training is the process of training the model on a massive dataset of text before it is used for any specific task.

It’s an absurd amount of text. GPT-3, for example, was trained on roughly 300 billion tokens drawn from a mix of sources such as filtered web pages (Common Crawl), books, Wikipedia, and more. The model reads all of this text and learns to predict the next token, over and over, adjusting its weights with each example.

Through this process, the model picks up patterns at every level of language. It learns spelling and grammar (how words and sentences are typically formed). It learns facts (associations between entities and their properties, like “Paris is the capital of France”). It learns reasoning patterns (if A implies B and B implies C, then A implies C). It learns style and tone (how a legal document sounds different from a text message). All of this comes from the statistical regularities in the training data.

One important distinction, though, is that the model is not simply memorizing text. Most of what makes it useful is generalization, the ability to apply learned patterns to new inputs it has never seen before. When you ask it a question it has never encountered verbatim, it can still produce a reasonable answer because it has learned the patterns behind how questions like that tend to be answered. That said, models can and do memorize some sequences, especially ones that appear frequently in the training data. It is best to think of it as mostly pattern-learning with some memorization mixed in.

It is also worth briefly mentioning how the model sees text. As we covered in the building blocks section, models do not operate on whole words. They use subword tokenization schemes like Byte Pair Encoding (BPE) to break text into smaller pieces:

• “unbelievable” might become [“un”, “believ”, “able”] (like we saw earlier)

• “ChatGPT” might become something like [“Chat”, “G”, “PT”] (the exact split depends on the tokenizer)

This lets the model represent unfamiliar or rare words by composing them from smaller pieces it has seen before.

Here is what to keep in mind heading into Act 4:

• Pre-training produces a general-purpose language model.

• It can do many things (answer questions, write code, summarize text, translate languages), but it is not specifically optimized for any of them.

• It is an incredibly capable autocomplete engine. However, you might ask it a question, and it might continue the text as if you were writing a webpage, generating more questions instead of an answer. Turning this raw capability into a helpful assistant requires additional training. That is where fine-tuning and RLHF come in.

Image Source: Deep (Learning) Focus

Act 4: From GPT to LLMs

Scaling Laws: Bigger Is (Predictably) Better

GPT-1 started as a proof-of-concept. It showed that a decoder-only Transformer could learn useful language patterns simply by predicting the next word in a sentence. At 117 million parameters and 7,000 books’ worth of data, it was a solid achievement for 2018, even if it wasn’t a total game-changer yet.

What followed was a bit of a turning point for the industry: the discovery that AI performance improves predictably as you increase the model size, data, and training time.

This was formalized in a 2020 paper from OpenAI called “Scaling Laws for Neural Language Models” (Kaplan et al.). The key finding was that the model’s loss (how bad its predictions are) decreases as a power law along three axes:

• Model size (number of parameters)

• Dataset size (number of tokens trained on)

• Compute (total amount of training computation)

What surprised people was how smooth and predictable this relationship is. You can plot model size on one axis and performance on the other, and it forms a clean curve. This means you can roughly forecast how good a model will be before you finish training it, just by knowing how big it is and how much data and compute you are throwing at it. It is not a guarantee, but it held up well enough to guide billions of dollars of investment.

The GPT family illustrates this scaling trajectory clearly:

ModelYearParametersNotable MilestoneGPT-12018117MProof of conceptGPT-220191.5BDeemed “too dangerous to release” at the timeGPT-32020175BFew-shot learning emergesGPT-42023UndisclosedMultimodal (text + images)

The important thing to notice is that there was no fundamental architectural change between GPT-1 and GPT-3. It is the same decoder-only Transformer, trained with the same next-token prediction objective. The difference is scale. More parameters, more data, more compute. And with that scale came capabilities that nobody explicitly programmed.

GPT-2 could generate surprisingly coherent paragraphs. GPT-3 could do things like answer questions, write code, and translate between languages, all without being specifically trained for those tasks. You could show it a few examples of a task (this is called few-shot learning), and it would figure out the pattern and apply it. This ability appeared as the model got bigger. It was not there in GPT-1, barely there in GPT-2, and clearly present in GPT-3.

But scale alone does not make a model helpful. A scaled-up GPT is still fundamentally an autocomplete engine. It predicts what text is likely to come next, not what text would be useful to a human. Getting from “impressive autocomplete” to “helpful assistant” required a different kind of training.

Fine-Tuning and RLHF: From Autocomplete to Assistant

Image Source: Deep (Learning) Focus

Imagine you have a friend who has read the entire internet. They know an extraordinary amount about everything. But if you ask them a question, instead of answering it, they just continue your sentence as if they were writing the rest of a webpage. You ask “What is the capital of France?” and they respond with “What is the capital of Germany? What is the capital of Spain?” because on the internet, questions are often followed by more questions.

That is what a raw pre-trained GPT model is like. It has the knowledge, but it has not learned how to be helpful.

The fix comes in two stages.

Stage 1: Supervised Fine-Tuning (SFT)

The first stage is conceptually simple. You hire human annotators and have them write examples of ideal behavior. They write prompts (“Explain quantum computing to a 5-year-old”) and then write high-quality responses. You then train the model on these examples so it learns to imitate the style, format, and helpfulness of the human-written responses.

This is called Supervised Fine-Tuning (SFT). It uses the same training process as pre-training (predict the next token, compute the loss, backpropagate), but the training data is now curated conversations rather than raw internet text.

SFT gets you a long way there. After this stage, the model starts responding to questions with actual answers instead of continuing text. But it still has problems. It might give plausible-sounding but wrong responses. It might be unnecessarily verbose. It might produce outputs that are technically correct but not what the user actually wanted. These are subtle quality issues that are hard to capture with simple “predict the next token” training.

Stage 2: RLHF (Reinforcement Learning from Human Feedback)

Image Source: Deep (Learning) Focus

This is the innovation that made ChatGPT feel different from everything before it.

The idea, introduced by OpenAI in their InstructGPT paper (Ouyang et al., 2022), is to train the model using human preferences rather than human-written examples. Here is how it works:

1. Generate multiple responses. For a given prompt, the model generates several different responses.

2. Humans rank them. Human raters look at the responses and rank them from best to worst. Which response is most helpful? Most accurate? Most safe? These rankings capture subtle quality differences that are hard to specify in a rule.

3. Train a reward model. Using those rankings, you train a separate model (called a reward model) to predict which responses humans would prefer. The reward model learns to assign higher scores to responses that humans rated highly.

4. Fine-tune with reinforcement learning. Finally, you use reinforcement learning (specifically an algorithm called PPO, or Proximal Policy Optimization) to adjust the language model so it generates responses that score highly with the reward model. The language model is essentially learning to produce outputs that humans would prefer.

In simple terms: SFT teaches the model what a good response looks like by showing examples. RLHF teaches the model why one response is better than another by learning from human preferences. Together, they turn a raw language model into something that feels like a helpful, conversational assistant.

This is a big part of why ChatGPT felt like such a step change when it launched in November 2022. The jump was not only about model size. A large part of the “ChatGPT feel” came from post-training with human feedback. OpenAI describes ChatGPT as a sibling model to InstructGPT, which was trained with RLHF to follow instructions. The model had been trained to care about what humans want, not just to predict the next word.

It is worth noting that RLHF is not the only approach. Anthropic (the company behind Claude) has developed related techniques called RLAIF (Reinforcement Learning from AI Feedback) and Constitutional AI, which supplement or replace human feedback in parts of the process. Meta and Google have also explored related alignment approaches. The details differ, but the core idea is the same: additional training after pre-training to make the model more helpful, harmless, and honest.

Reasoning: The “Thinking” Part

Remember the title of this post? “ChatGPT Doesn’t Think. It Pays Attention.” That is true in a fundamental sense. Everything under the hood is still attention and next-token prediction. But recent developments have pushed these models into territory that looks a lot like thinking, and it is worth understanding how.

Chain-of-Thought Prompting

In 2022, researchers at Google published a paper (Wei et al.) showing that if you prompt a language model to “think step by step,” its performance on complex reasoning tasks improves dramatically. This technique is called chain-of-thought (CoT) prompting.

For example, if you ask a model, “If a store has 3 apples and someone buys 2, then the store gets a delivery of 7 more, how many apples does the store have?” and just ask for the answer, it might get it wrong. But if you prompt it to work through the problem step by step, it performs much better because each intermediate step (3 - 2 = 1, then 1 + 7 = 8) becomes part of the generated text, and the model can attend to its own reasoning as it goes.

One way to think about this is that the model is doing its “reasoning” in the form of text it can read back. The intermediate steps are tokens, and the model attends to those tokens when generating the next ones. It is still next-token prediction, but applied to a reasoning trace that the model itself is producing. From this perspective, it is paying more attention rather than thinking in the way humans do.

Reasoning Models

Image Source: MIT CSAIL

More recently, companies have taken this idea further by training models specifically to produce extended reasoning chains before giving a final answer. OpenAI’s o1 and o3 models and Anthropic’s Claude with extended thinking are examples.

These models are trained (often using reinforcement learning) to generate long internal reasoning traces. When you ask a complex question, the model might produce hundreds or thousands of tokens of step-by-step reasoning before arriving at an answer. The final answer tends to be significantly more accurate than what a model would produce if it tried to answer immediately.

The underlying mechanism is still attention and token prediction. But by training the model to produce intermediate reasoning steps, you effectively give it more “compute” at inference time. Instead of going directly from question to answer in a single pass, the model takes many steps, and each step can attend to all previous reasoning. It is similar to giving someone scratch paper for a math test versus asking them to do it all in their head.

So Does It Think?

This is one of the most debated questions in AI. The honest answer is it depends on what you mean by “think.”

If thinking means manipulating symbols step by step to arrive at a conclusion, then yes, reasoning models do something that looks like that. If thinking requires genuine understanding, consciousness, or intentionality, then no, these models are still doing sophisticated pattern matching over tokens.

What is clear is that these techniques produce better results on complex tasks. Whether you call it “thinking” or “structured token prediction” is partly a philosophical question. The practical reality is that the same core mechanism (attention + next-token prediction) can be steered to produce remarkably capable reasoning behavior through the right training and prompting techniques.

And that ties back to the title; even when the model “thinks,” it is paying attention.

Emergent Abilities and the LLM Paradigm

Something unexpected happened as language models got bigger. They started doing things that nobody explicitly trained them to do.

These are called emergent abilities (Wei et al., 2022), and they are one of the more surprising and interesting phenomena in modern AI. An emergent ability is a capability that is not present in smaller models but appears once the model reaches a certain scale. On some benchmarks, it can look like the ability turns on suddenly, almost like a phase transition, even though the underlying improvements may be more gradual than they first appear.

Examples include:

• Few-shot learning. Show the model a few examples of a task it has never seen, and it figures out the pattern. GPT-3 could do this. GPT-2 mostly could not.

• Chain-of-thought reasoning. The step-by-step reasoning we just discussed only works well in models above a certain size.

• Code generation. Large models can write working code from natural language descriptions, even though they were trained primarily on text, not specifically on programming tasks.

• Translation between languages the model saw relatively little of during training.

• Arithmetic on numbers the model has never seen before (up to a point).

What makes this surprising is that none of these abilities were explicitly part of the training objective. The model was just trained to predict the next token. But at sufficient scale, next-token prediction apparently requires learning representations so rich that these capabilities come along for free.

This has led to a paradigm shift in how people think about AI systems. Before LLMs, the standard approach was to build a separate model for each task: one model for translation, one for summarization, one for question answering, and one for sentiment analysis. Each one required its own architecture, training data, and engineering effort.

The LLM paradigm is different. You train one very large model on a general objective (predict the next token), then adapt it to specific tasks through prompting, fine-tuning, or RLHF. This is why companies like OpenAI, Anthropic, Google, and Meta have invested so heavily in building larger and larger models. The returns from scale have been extraordinary so far.

Of course, whether this paradigm will continue to hold is an open question. There is active debate about whether emergent abilities are truly sudden or just artifacts of how we measure them. There are real limitations to what these models can do (they can still hallucinate confidently, struggle with certain types of reasoning, and make basic factual errors). And the cost of training and running these models is enormous, raising questions about sustainability and access.

But the trajectory from 2017 to today is remarkable. A single architecture, the Transformer, built on a single mechanism, attention, has gone from improving machine translation to powering the most capable AI systems ever built. And it all started with a paper titled “Attention Is All You Need.”

Act 5: Tying It All Together

We have covered a lot of ground. Let’s trace the full journey one more time.

It started with a problem. RNNs processed language one word at a time, struggling with long sequences and unable to take advantage of modern parallel hardware. Attention mechanisms helped, but they were bolted onto the sequential architecture rather than replacing it.

In 2017, a team of researchers at Google tried to find out what would happen if we dropped the sequential part and relied only on attention. The Transformer they built replaced recurrence entirely with self-attention, allowing every token to attend to every other token in a single step. With multi-head attention, they gave the model multiple perspectives on every word. And positional encodings preserved word order. On the other hand, residual connections and layer normalization kept training stable, and the final result was a model that could process entire sequences in parallel, learn long-range dependencies, and train significantly faster than anything before it.

Then came GPT, simplifying that architecture further. Researchers at OpenAI removed the encoder and cross-attention, relying only on the decoder stack, and trained it to predict the next token. The model also learned grammar, facts, reasoning, style, and more. Over time, we learned that as we scale up with more parameters and more data, we start to get capabilities that were never explicitly programmed. Finally, adding in fine-tuning and RLHF and we get a model that stops being a fancy autocomplete and starts acting like a helpful assistant.

Every time you talk to ChatGPT, Claude, Gemini, or any other AI assistant, this is what is happening under the hood. The input text gets turned into tokens, those tokens are embedded into vectors along with their positional information. Then they flow through layers of attention where every token attends to every other token, gathering context from across the entire input. Then the model predicts the next token. And the next. And the next. One token at a time, with each subsequent token informed by everything that came before.

It is attention, all the way down.

Writing this post over the past few weeks has been a big learning experience for me. The thing that surprised me most is how simple the core ideas are after I started to understand them. None of the individual components is complicated on its own; what makes them powerful is how they compose together and what happens when you scale them up.

I also came away with a deeper appreciation of the researchers behind this work. It is easy to look at the logic and think it is obvious, but it took years of work, building on decades of research in neural networks, sequence modeling, computing, and optimization, to arrive at something that looks this clean.

If you want to go deeper, attached below are resources that I found most valuable while learning the material. And if this is the first article you are reading in this series, be on the lookout for the other posts on AI agents and coding tools like Claude Code or Codex, building on the foundation laid here. Thanks for reading. I hope this helped demystify what is going on inside the AI tools you use every day. If you have questions, feedback, or corrections, I would love to hear from you.

ChatGPT doesn’t think. It pays attention. Now you know how.

Resources

Interactive Tools

• Transformer Explainer (Georgia Tech Polo Club) - Interactive visualization where you can watch attention and data flow through a Transformer in real time. Spend an hour with this.

• Karpathy’s minGPT - A full GPT implementation in about 300 lines of Python.

• Karpathy’s nanoGPT - More recent and minimal GPT training repo. Pairs well with his YouTube walkthrough linked below.

• Transformer Explainer source code - Source code for the Transformer Explainer tool.

• Transformers in Action code - Code repository for the Transformers in Action book.

• AnimatedLLM: Explaining LLMs with Interactive Visualizations, AnimatedLLM - Understanding the mechanics of LLMs (Includes text generation and training)

• LLM Visualization - Interactive walkthrough of the internals of various LLMS such as nano-gpt, GPT-2, and GPT-3.

The Original Paper

• “Attention Is All You Need” (Vaswani et al., 2017) - The paper that started it all. Worth reading even if you skip the math.

Video Explainers

• 3Blue1Brown: Large Language Models explained briefly - Short visual intro to LLMs, chatbots, pretraining, and tokens.

• 3Blue1Brown: But what is a GPT? Visual intro to Transformers - Visual explanation of how GPT works from the ground up.

• 3Blue1Brown: Attention in Transformers, visually explained - Step-by-step walkthrough of the attention mechanism.

• 3Blue1Brown: Visualizing Transformers and Attention (full talk) - Grant Sanderson’s full conference talk. More detailed than the shorter videos.

• Andrej Karpathy: Let’s build GPT from scratch - Two hours of building a GPT from scratch in code. Long but worth it.

• Andrej Karpathy: Deep Dive into LLMs like ChatGPT - General audience deep dive into LLM technology.

• StatQuest: Transformer Neural Networks, Clearly Explained - Methodical, step-by-step walkthrough.

• IBM Technology: How Large Language Models Work - Accessible overview from IBM.

• Polo Club: Transformers Explained Visually - Video companion to the Transformer Explainer tool.

• Umar Jamil: Attention is All You Need (including math) - Detailed walkthrough of the paper with math, inference, and training.

• Leon Petrou: Transformers, explained - Another visual explanation of the architecture.

Further Reading

• Jay Alammar: The Illustrated Transformer - Visual guide to the Transformer with diagrams for every component.

• Sebastian Raschka: Understanding Self-Attention from Scratch - Code-first walkthrough of self-attention.

• Lilian Weng: The Transformer Family Version 2.0 - Comprehensive survey of Transformer variants and improvements.

• Lilian Weng: Attention? Attention! - Thorough overview of attention mechanisms from basics to advanced variants.

• Christopher Olah: Understanding LSTMs - Classic post explaining LSTMs with diagrams. Good for understanding what came before Transformers.

• Amirhossein Kazemnejad: Transformer Architecture - The Positional Encoding - Deep dive into positional encoding.

• Transformers in Action (Nicole Koenigstein, Manning) - Hands-on book for implementing Transformers.

• MIT 6.390: Introduction to Machine Learning - Transformers - Lecture notes on Transformers from MIT.

• Google ML Crash Course: Introduction to LLMs - Google’s structured intro to large language models.

• Hugging Face LLM Course: How do Transformers work? - Part of Hugging Face’s free LLM course.

• Understanding AI: Large language models, explained with a minimum of math and jargon - Plain-language LLM explainer.

• Miguel Grinberg: How LLMs Work, Explained Without Math - Another no-math explainer focused on intuition.

• freeCodeCamp: A Beginner’s Guide to Large Language Models - Beginner-friendly intro.

• IBM: What is a Transformer Model? - IBM’s overview of Transformers.

• IBM: What are Large Language Models? - IBM’s LLM overview.

• Stanford: AI Demystified - LLMs - Stanford’s introductory guide.

• MIT News: Large language models use a surprisingly simple mechanism to retrieve stored knowledge - Research on how LLMs retrieve facts internally.

Key Papers

• GPT-1: Improving Language Understanding by Generative Pre-Training (Radford et al., 2018)

• GPT-2: Language Models are Unsupervised Multitask Learners (Radford et al., 2019)

• GPT-3: Language Models are Few-Shot Learners (Brown et al., 2020)

• GPT-4 Technical Report (OpenAI, 2023)

• InstructGPT: Training language models to follow instructions with human feedback (Ouyang et al., 2022) - The RLHF paper behind ChatGPT.

• Scaling Laws for Neural Language Models (Kaplan et al., 2020)

• Chain-of-Thought Prompting Elicits Reasoning in Large Language Models (Wei et al., 2022)

• Emergent Abilities of Large Language Models (Wei et al., 2022)

• Constitutional AI: Harmlessness from AI Feedback (Bai et al., 2022) - Anthropic’s alignment approach.

• Llama 2: Open Foundation and Fine-Tuned Chat Models (Touvron et al., 2023)

• Neural Machine Translation by Jointly Learning to Align and Translate (Bahdanau et al., 2014) - The paper that introduced attention.

• Sequence to Sequence Learning with Neural Networks (Sutskever et al., 2014) - The encoder-decoder paper.

• Deep Residual Learning for Image Recognition (He et al., 2015) - Residual connections.

• Layer Normalization (Ba et al., 2016)

• RoFormer: Enhanced Transformer with Rotary Position Embedding (Su et al., 2021) - RoPE, used in many modern LLMs.

• Efficient Transformers: A Survey (Tay et al., 2020)

• FlashAttention: Fast and Memory-Efficient Exact Attention (Dao et al., 2022)

• Transformer Explainer: Interactive Learning of Text-Generative Models (Cho et al., 2024)

• Are Emergent Abilities of Large Language Models a Mirage? (Schaeffer et al., 2023)

“Even if it don’t benefit your goals, do some push-ups…” - Kendrick Lamar

Image Source: XKCD

Introduction

Around the middle of last year, I decided to go on a bulk as a way to gain muscle. This went on for a couple of months, and I gained about 25 pounds. I also made a few mistakes along the way: I stopped running and did very little cardio apart from soccer. I got busy at work and let my regular weight training slide.

By September, I was 230 pounds and feeling less and less like my normal self. I decided the bulk was enough — it was time to cut.

Over the next couple of months, I followed a disciplined cutting plan and learned a lot that could be helpful for anyone in a similar position. The biggest lesson was learning how to build healthy habits that are actually sustainable, rather than riding the ups and downs of gaining and losing weight.

Before I get into strategies and recommendations, I want to be upfront about something: the subject of weight loss and diet can be tricky. I strongly believe everyone can pursue better health regardless of their current size; whether that means losing weight, gaining weight, or maintaining. Wherever you are right now is a valid starting point, not a moral failing. My goal here is to share what worked for me and offer a useful starting point you can adapt to your own goals.

And yes, I’m going to lean into getting lean and building muscle. So consider this a practical guide for anyone like me who wants to get lean (and maybe a little shredded 🤞🏾) before GTA 6 comes out (whenever that is)… and actually keep those gains.

Setting the Stage (or Arranging the Gym)

Image Source: Andy Anderson Cartoons

Before we jump into strategies and plans, it’s worth establishing some concrete ideas: what it actually means to be lean, why health and sustainability matter more than extremes, and how to stay mindful of social media influence.

Why Bother Getting Lean or Building Muscle?

The simple answer is: it usually makes people healthier, and often more confident in their bodies (and yes, sometimes more attractive to others). But that’s vague, so here’s what I actually mean.

Health

Most bodies function best in a “middle zone.” That is, not too heavy or too light, but in a range where energy, hormones, mood, sleep, and performance tend to be more stable.

• On the higher end, carrying a lot of extra body weight can make everyday movement harder, reduce endurance, and increase strain on joints. It can also make it easier to drift into a lifestyle where you move less, recover worse, and become more at risk for health issues (especially related to the heart).

• On the lower end, getting extremely lean can come with its own problems. When body fat gets too low (especially for long periods), the body can start treating life like a survival situation: energy drops, recovery gets worse, cravings go up, and brain fog can kick in. Hormones can also take a hit, and women may lose their menstrual cycle, while men can see testosterone drop. Plus, people at this level can feel cold, tired, and hungry all the time.

That’s why the goal here isn’t to be “as lean as possible.” It’s to get lean enough to feel good, move well, perform well, and stay healthy.

This also means we don’t need to train like professional athletes. Basic guidelines like regular movement and aiming for ~150 minutes per week of moderate-intensity cardio are a great baseline. And adding strength training on top of that tends to compound the benefits, which takes us to our next section.

Muscle

Muscle matters because it’s the foundation of strength and stability (joints, posture, balance), movement (walking, stairs, lifting groceries, playing sports), and resilience (recovering from injury and staying capable as you get older).

Too little muscle is limiting. Too much muscle, especially when it’s pushed with performance-enhancing drugs, can come with tradeoffs, too. For most of us, the sweet spot is being strong enough to live well and move confidently, without making fitness our entire identity (and without sacrificing the rest of our lives to maintain it).

Attractiveness

Attraction is personal and cultural, and nobody owes anyone a certain look. That said, people often respond positively to signs of health such as energy, confidence, good posture, and the ability to move well. My take is that if you take care of your health, you tend to show up better in life, and that often translates socially, too.

So the goal of getting lean and building muscle, at least for me (and hopefully for others too), is to live longer, feel better day to day, be capable, and enjoy more of life without living at the extremes.

Social Media and TV

Image Source: exercise myths | Man Bicep

Now that we’ve talked about why someone might want to get lean or build muscle, it’s worth acknowledging the media environment we’re working in.

The modern fitness “baseline” is warped. Movies, Instagram, and YouTube are full of people who look shredded, dry, and jacked year-round—and it quietly trains our brains to think that’s what “normal fit” looks like. But much of what we see is a peak look, not an everyday one, and it often comes with hidden methods and costs.

A few things that routinely get left out of the highlight reel:

• Curation + selection bias: We are mostly seeing people with great genetics for building muscle, years of experience, insane training regimens, and a strong incentive to look exceptional on camera.

• Camera tricks: Lighting, angles, a pump, and posing can make someone look dramatically leaner and bigger than they do walking around on a random Tuesday.

• “Peak Week” Physiques: Some of the physiques that we see that look incredible are achieved temporarily through aggressive dieting, dehydration, and other short-term tactics that don’t feel great to maintain.

• Editing: Everything from filters to retouching to post-production can make bodies look more “superhero” than human.

This matters because it changes the comparison game. Research consistently finds that exposure to idealized physiques can increase body dissatisfaction and appearance comparison.

Similarly, while not everyone with an impressive physique is using performance-enhancing drugs (some people are dedicated and genetically gifted), enough people are using them (or going to extreme, temporary, photo-shoot preparation) that it is hard to know what you’re looking at. At the same time, PEDs are not harmless; they’re associated with real health risks, including cardiovascular and psychiatric effects. And using these drugs gives people a real boost. One analogy (popularized by Coach Greg Doucette, a polarizing but useful voice in the fitness space) is that training naturally is like building a house with one worker, while using PEDs is like adding several other people to join the job site. The progress can look completely different, and so does the cost.

For a deeper look at how normalized this is, the documentary Bigger, Stronger, Faster is a good cultural snapshot into the fitness space, especially in America, and how blurry the line gets between aspiration and deception.

Some actors who have played superhero roles have openly acknowledged how brutal it can be to prepare for such a role.

• Zac Efron: has described his Baywatch look as difficult to attain and unsustainable, sharing about the depression/insomnia and the extreme measures involved.

• Kumail Nanjiani was unusually honest about the resources behind his transformation for Eternals, including a full year, top trainers, elite nutrition support, and basically turning fitness into a job.

• Chris Hemsworth has also spoken publicly about the intensity of building and maintaining a Thor-level physique (and more recently has emphasized a shift toward recovery and sustainability over extremes).

Another useful lens here is Derek from More Plates More Dates, who spends a lot of time talking about how common PED use is in certain fitness and entertainment circles and why transparency matters. Whether or not you agree with his style, he’s part of a broader pushback against fake-natty marketing and unrealistic expectations.

There’s also an academic side to this. A growing body of research links heavy exposure to idealized fitness content with things like obsessive food/exercise behaviors and muscle dysmorphia (sometimes called “bigorexia”), especially in young men comparing themselves to hyper-muscular bodies online.

And for women, the social-media pressure to be extremely lean can overlap with low energy availability/RED-S (a well-studied syndrome where chronic under-fueling can disrupt hormones, recovery, and overall health).

My point isn’t that we should not watch fitness content. It’s that we should use fitness content for ideas and lessons, and not for identity. If you’re training naturally (and even if you aren’t), it’s helpful to be careful about who you compare yourself to. A body you can maintain while sleeping well, performing optimally, and enjoying your life is a better target than the most flattering look that might require your whole world to revolve around staying shredded.

Can We Finally Start Getting Lean? (Yes)

Nutrition: It Always Starts With Food

Like the XKCD comic above, our metabolisms follow the laws of thermodynamics. When we eat, our bodies pull energy from food. If we eat more than we need, the extra gets stored as fat. If we eat less, the body taps into fat stores, glycogen, and sometimes muscle for energy. So to gain weight, we need to eat more than we burn; to lose weight, we need to eat less.

It is really that simple at the highest level, but let’s break it down some more.

How Weight Loss (and Gain) Actually Works

Each of us burns a baseline amount of energy every day to keep our bodies running: the brain, digestive system, heart, and everything else. This is called your basal metabolic rate (BMR). On top of that, any movement or exercise burns additional energy. The total of these two is your total daily energy expenditure (TDEE).

Hence, the formula is:

TDEE = BMR + Energy from Activity

• Eat consistently below your TDEE → lose weight

• Eat consistently above your TDEE → gain weight

• Eat around your TDEE → maintain

Quick Start (14 days)

• Weigh daily and track a 7-day average

• Pick cut (–300 to –500/day) or lean bulk (+150 to +300/day)

• Hit protein daily + lift 2–4x/week

• Walk 7–10k steps/day

• After 14 days, adjust by 150–250 calories/day based on the trend

We measure this energy in calories. As a rough benchmark, about 3,500 calories equals one pound of body fat. So a daily deficit of 500 calories generally leads to about one pound of weight loss per week. A surplus of the same amount leads to about a pound gained.

Not All Calories Are Created Equal

Image Source: Fuel Running

While the math of calories in vs. calories out drives weight change, what you eat matters a lot for how you feel along the way. Our food contains three macronutrients, and each behaves differently.

• Protein is digested slowly and requires more energy to process, which keeps you full longer. It’s also essential for preserving muscle during a cut. This is the one to prioritize.

• Carbohydrates digest quickly and provide near-instant energy. But eating a lot of them (particularly highly refined carbs) at once can spike your energy, followed by a crash (and sometimes hunger or sleepiness).

• Fats are more calorie-dense (9 calories per gram as opposed to 4 for protein and carbs) and tend to make food taste better, but they add up fast.

Fiber is also worth mentioning as high-fiber foods take longer to digest, support gut health, and help keep you full.

This is why calorie density matters so much in practice. Calorie-dense foods like cookies, soft drinks, and cake go down quickly and barely register as filling. Less calorie-dense foods like vegetables, fruits, and lean proteins fill you up with far fewer calories. When you’re trying to lose weight, leaning toward the latter makes the whole process dramatically easier.

Making it Practical

Image Source: Glasbergen Cartoon Services

So here’s the part we can apply repeatedly.

1. Know your numbers (roughly)

• Find your BMR using an online calculator (I like this one from sailrabbit, but there are many others, like this simpler one from MyFitnessPal).

• Estimate your daily activity calories using a fitness watch (Apple Watch, Fitbit, etc.), or you can compute your BMR and TDEE from an online calculator like this one from calculator.net.

These numbers are estimates, but that’s fine; all calorie measurements are. The point is to have a rough idea of your daily calorie burn and a working target.

2. Pick Your Goal and a Realistic Pace

If you’re cutting (losing fat):

• Aim to lose about 0.5-1.0% of your bodyweight per week (slower is generally better because it is more sustainable). For most people, this amounts to a daily deficit of roughly 300-500 calories.

If you’re lean bulking (gaining weight/building muscle without getting fluffy):

• Aim to gain about 0.25-0.5% of your bodyweight per week. A surplus of roughly 150-300 calories per day is a good starting point.

Whatever your goal is, try to avoid extremes. For instance, a 1,000-calorie deficit would lead to faster weight loss (roughly 2 pounds per week), but it would feel miserable and hard to sustain.

3. Use the Scale as a Feedback Loop

• Weigh yourself daily, ideally in the morning under the same conditions.

• Ignore individual days. Your weight can swing 2-3 pounds overnight based on hydration, inflammation, glycogen levels, and what you ate. Look at the 7-day average instead.

• After every two weeks, adjust based on the trend:

  • If you’re not losing on a cut: Reduce food slightly or add more daily exercise (for instance, ~150-250 calories/day or add ~2-3k steps/day).

  • Losing too fast or feeling awful: Add some food back (~150-250 calories/day).

  • Not gaining on a bulk: Add ~150–250 calories/day (a snack or bigger portions).

4. Hit the Big Rocks

If you do nothing else, focus on these:

1. Protein: Aim for roughly 0.7-1.0g per pound of your goal bodyweight per day. This is especially important while cutting, as protein helps keep you full, supports muscle repair, and helps ensure the weight you lose is mostly fat.

2. Fiber and volume: Fill your plate with fruits, vegetables, beans, and whole grains. These foods physically fill you up without packing a lot of calories.

3. Watch liquid calories: Sugary drinks, juices, and other “easy calories” add up fast and don’t satisfy hunger. This is one of the simplest places to cut.

4. Keep strength training: Lifting while cutting sends your body the signal to hold onto muscle. Without it, you risk losing muscle along with fat.

5. Make Things Sustainable

The best diet is the one you can stick to long-term. The trick is to keep the foods you enjoy while swapping portions or ingredients so they still fit your target.

There are also a ton of low-calorie cookbooks and recipes, as well as ways to prepare regular foods in low-calorie ways (for instance, baking instead of frying chicken). The key is building a rotation of meals you look forward to eating, not ones you tolerate.

6. Plan for the Real World

One big meal with friends won’t derail your goals. When we know a big meal is coming, it is best to plan for it:

• Keep earlier meals lighter and protein-focused.

• Get some extra steps or exercise in.

• Enjoy the meal without guilt.

• Get back to your normal plan the next day.

Some days you’ll eat more, some days less. What matters is the trend over weeks, not perfection on a single day. Similarly, if your energy starts to drop, eat more food. Sometimes, when we are in a deficit, we notice we are having trouble performing or focusing, and we consistently feel drained. In times like this, it is best to eat a bit more until we find the sweet spot where we’re still in a deficit but can function like we normally would.

7. Think in Long Timeframes

Don’t try to lose 25 pounds in three months. That pace is aggressive and could be hard to sustain, especially depending on your current body fat level. A year, for instance, is a better timeframe. That is 2-3 pounds a month, which is steady, manageable, and far more likely to stick. A quick transformation usually doesn’t last if it requires unsustainable habits. However, if you want an initial motivation hit, you can start aggressively and then move into a maintenance pace that doesn’t destroy your life.

Gaining weight, on the other hand, is generally easier since you need to find foods you enjoy and eat slightly more of them.

What I Did (Full Transparency)

I want to be upfront: my approach was more aggressive than what I’d recommend for most people. When I was bulking, I didn’t track anything and just ate however I felt. When I started cutting, I used the app Cronometer and connected it to Apple Health so I could see my total daily calorie burn alongside my intake. And I’ve been logging every single thing I’ve eaten, without exception, over the past 5 months.

That level of tracking worked for me, and I genuinely enjoyed being able to tell exactly what I ate on a random Tuesday in December. But it’s not necessary for everyone, and for some people, that degree of monitoring can become unhealthy or unsustainable. The approach above (rough targets + scale trend + enjoyable foods) will get most people there without logging every bite.

An Example Day (Cut)

Here’s roughly what a typical cutting day looked like for me. It’s an example of what ~1,900–2,100 calories can look like when you prioritize protein and volume (i.e., lots of filling, lower-calorie foods like fruits and vegetables). I also aimed for ~185–220g of protein/day based on my body weight at the time and my goal of minimizing muscle loss during the cut.

The Bottom Line

You don’t need perfect macros or a perfect meal plan. Pick a goal, track the trend, adjust slowly, and build a menu you actually enjoy. The goal is a way of eating you can maintain so that when you get lean, you stay lean.

A few extra notes:

Cheat Days

Image Source: Kate Taylor

Cheat days work for some people. For me, they didn’t. When I gave myself a weekly cheat day, I’d go nuclear and eat 6,000–7,000 calories, basically undoing a week of progress (I like food a lot).

What worked better was building a way of eating that works every day, while still leaving room for foods I enjoy. If you like the idea of flexibility, a few alternatives that tend to be easier to manage than a full cheat day are:

• Planning treats (for instance, a small dessert every day)

• A single high-calorie cheat meal instead of eating high-calorie foods for an entire day

• An approach where your meals are split 80/20, or 90/10, such that most of them are on the plan, and a few are purely for joy

Weight-loss Medications

I’m not a doctor, but it’s hard to ignore how common weight-loss medications have become. I do think many people can lose (or gain) weight without medication, but I also know that isn’t true for everyone, and there are cases where medical support can make a real difference. It’s also worth noting that these medications work in ways closely related to what we’ve discussed here. They mimic hormones that signal fullness, reducing appetite and leading to less food intake and, correspondingly, weight loss. In other words, it’s still an energy balance, just with pharmacological help on the appetite side.

If you talk with your doctor and they think a medication is appropriate and safe for you, I don’t think there’s anything wrong with using that tool. I’d encourage honesty about it, especially if you’re sharing advice publicly.

In general, I think of these meds the same way I think about other “boosts” in life. Support can help us gain traction. For instance, I got a boost writing this post from other articles, research, videos, and AI tools like ChatGPT and Claude. These boosts didn’t entirely replace the work, but they helped me do it more efficiently.

My only caution is that even with a boost, habits still matter. So, keep protein high, do strength training, and be mindful of potential side effects with your doctor (especially so you don’t lose more muscle than you need during a cut). Some of the more common medications, like GLP-1 agonists, can accelerate muscle loss alongside fat loss, which makes protein intake and strength training even more important if you’re using them.

Exercise: Move in Ways You’ll Actually Keep Doing

Image Source: Red Bubble

Just like nutrition, the best exercise routine is the one you’ll actually stick to. This means we do not need a perfect program (such a program actually doesn’t exist). We also do not need to train like professional athletes, or make training the entirety of our lives. What we need is something sustainable that we can repeat week after week, month after month, ideally for several years.

Diet drives most of fat loss, but training determines what that fat loss looks like. Two people can lose the same amount of weight, but the one who lifts and stays active will likely retain more muscle, have better posture, and feel stronger. If nutrition is the steering wheel, exercise is the engine.

But before I share more, let’s start with a quick start.

Quick Start (14 days)

• Lift 2-4x/week (full body is simplest)

• Do cardio 2x/week (~150 minutes of moderate intensity cardio per week) or hit 7-10k steps/day

• Pick one fun activity that gets you moving that you can do weekly (good examples are soccer, tennis, dancing, pickleball, flag football, etc.)

• Track just two things: workouts completed and steps average

• Don’t miss twice in a row

• 14 days in, adapt and keep going

That’s enough to build momentum; everything else is refining.

Strength Training

Image Source: XKCD

If you’re trying to get lean and keep muscle (or build it), strength training is probably the single most important type of exercise for that goal. When you’re in a calorie deficit, your body is looking for energy wherever it can find it. Lifting sends a clear signal to your body that your muscles are being used, so it should not burn them for fuel. Without that signal, you risk losing muscle along with fat, and the result is a smaller but softer version of yourself.

The good news is that you don’t need anything complicated. Once you pick a proven program and stick with it, you are well on your way. Here are a few solid options depending on your schedule:

2 days/week:

• Day 1 and 2: Squat or hinge + push movements + pull movements

3 days/week (full body):

• Day A: Squat or leg press + bench press + rows + accessories

• Day B: Deadlift or RDL + overhead press + pull-ups/lat pulldown + accessories

• Day C: Repeat A or B with slight variations

4 days/week (upper/lower split):

• Upper days: push + pull + shoulders + arms

• Lower days: squat or hinge movement + calves + core

These are simple guidelines, but there are various ways to approach strength training, and you can explore several options, gradually fine-tuning into what works best for you. Most of the strength and growth come from doing the basics consistently and getting better over time. A few things to keep in mind:

• Train hard: Showing up is step one, but progress and strength usually come from pushing yourself and working most sets close to failure. This could come in the form of progressive overload (gradually increasing weight, reps, or sets over time), or simply pushing yourself a tiny bit more than you did the previous session (for instance, holding the bar up a little longer than the previous sessions when doing a bench press).

• Form matters: To reduce the risk of injury, it is helpful to learn and maintain proper form while exercising. Ask others to watch your form and give you feedback, or use YouTube tutorials and guides. If the weight is so heavy that you start to lose form, decrease it and nail the form before progressing. This exercise library from Hevy is an excellent resource for finding different exercises and their correct form.

• Track your lifts: In a way most suitable for you, keep a log of your weights and reps. It’s hard to progressively overload if you don’t remember what you did last week. You can use an app like Stronglifts, or Strong, or even just a notes or spreadsheet app on your phone.

Image Source: XKCD

As a final note on strength training, I’ve talked to some of my friends who are women about lifting, and a common concern I hear is not wanting to get bulky or look masculine from it. I’ve heard this enough times that it’s worth addressing directly. And honestly, the concern is understandable. Gym culture can feel intimidating, and strength training spaces aren’t always as welcoming or comfortable for women as they should be.

The short answer: it’s extremely unlikely to happen. The “bulky” look most people picture typically requires levels of testosterone that women’s bodies don’t naturally produce. Men generally have much higher testosterone, which is a major driver of the muscle size people associate with a masculine physique. For the vast majority of women, strength training builds muscle in ways that make them stronger, leaner, healthier, and more toned; not bulky.

And when you do see extreme muscularity, it’s usually the result of years of highly dedicated training and, in many cases, performance-enhancing drugs—which, as we covered earlier, come with real tradeoffs and aren’t something most people should be using.

Cardio

Image Source: Red Bubble

Cardio gets a bad reputation in some lifting circles because people believe it kills your muscular gains. However, cardio is worth doing even if your primary goal is to look better. It improves your heart health and strength (the most important muscle in your body), endurance, mood, recovery, and day-to-day energy.

While strength training is non-negotiable for getting lean and keeping muscle (the focus of this post), I personally think cardio is the most important group of exercises for overall health and longevity. If I could only pick one, I’d choose cardio. I understand not everyone shares that perspective, but regardless of where you fall, having decent conditioning makes everything else in life easier.

A good baseline is the classic guideline of aiming for around 150 minutes per week of moderate-intensity cardio. You do not need to start here, though; you can gradually build up to it. And just like strength training, we get better at cardio by challenging ourselves each session to do a little more or go a little harder than before. Over time, the improvements compound.

We can also think of cardio in two buckets:

Low-intensity/daily movement

• These include walking, taking the stairs, parking farther away, pacing on phone calls, etc. Typically, these do not feel like exercise, but they add up.

• Aiming for 7–10k steps/day (or if you want a challenge, 12–15k steps) is one of the simplest and most effective things you can do.

• Low-intensity cardio burns calories without spiking hunger the way intense cardio can and generally doesn’t require recovery time.

Moderate-to-high intensity

• Running, cycling, swimming, rowing, sports, etc. Your goal in this bucket is to choose whatever you enjoy.

• These build real cardiovascular fitness, and you can steadily challenge yourself and improve.

Image Source: Red Bubble

Like we’ve highlighted throughout this post, enjoyment matters more than anything when choosing the type of cardio to do. If you hate running, don’t run. Walk, bike, swim, dance, hike, play basketball, join a rec league, skate. The best cardio is the one you’ll keep doing, especially when motivation wanes.

A practical starting point is:

• 2 sessions/week of 20–40 minutes (plus steps from walking)

• Daily steps + one weekly hard session from a sport you enjoy

One important note: if you’re lifting hard and also doing intense cardio, your recovery might start to suffer. In that situation, reduce your cardio intensity before cutting back on lifting. And as much as you can, keep walking.

Finding What You Actually Enjoy

I know I keep coming back to this, but it’s because it’s genuinely the most important piece of fitness advice I can offer. If you hate your workout, you’ll eventually stop doing it. It doesn’t matter how “optimal” the program is; what matters is your ability to do it consistently. And liking the workout is the first step.

• If you like lifting but hate cardio -> keep cardio simple (walking + a sport)

• If you like cardio but hate the gym -> lift twice a week and keep it short

• If you get bored easily -> rotate activities, but keep the weekly structure

Sports, in particular, are massively underrated as exercise. They generally don’t feel like working out because while doing them, you’re focused on playing, competing, and having fun. For me, soccer has been one of the most consistent forms of cardio in my life because I play with my friends, and I genuinely have fun anytime I’m on the pitch. I’ve also found that activities like dancing (choreography, ballet, partner dancing, etc.), yoga, group classes, and martial arts are great options. Doing these activities with friends, in a class, or on a team helps you stay consistent, as you rely less on personal motivation.

Image Source: Creative Canvas Shop

Speaking of the social aspect of training, I recently started going to the gym with my friends instead of going alone, and it has been a tremendous change. The camaraderie has been amazing. For instance, we recently celebrated one of our birthdays at the gym with donuts (cheat meal). We also challenge each other, and we’re all able to work out harder because we have people to provide extra support and spot each other. Below is a recent picture of some of us at the gym:

Here’s also something that might sound counterintuitive: it’s okay to keep your routine boring. Same workouts, same days, same structure, week after week. Boring is sustainable and becomes automatic. You do not need to reinvent your program every month. You only need to keep showing up and pushing a little harder over time.

Image Source: Go Comics

A Final Note on Training Hard

Effort matters, and there’s a real difference between showing up and going through the motions versus actually pushing yourself. The people who make noticeable progress are the ones who consistently bring intensity to their sessions.

But training hard doesn’t mean ignoring your body. Recovery is part of the process. And sleep is arguably the most important recovery tool we have, as it’s when our bodies actually repair and build muscle. A few other recovery basics:

• Rest days: Take 1–3 per week depending on your program and how you feel. Active recovery (a walk, light stretching) is great on rest days.

• Deload weeks: Every 4–8 weeks, it is helpful to consider a lighter week where you reduce volume or intensity. This lets your body catch up and often leads to better performance in the following weeks.

• Listen to persistent pain: Soreness is normal, especially when you train hard. Sharp pain that doesn’t go away, however, is a signal to back off and address it before things worsen. If pain persists, see a physical therapist or a doctor—it’s worth it.

Overtraining and injury are big threats to consistency. And consistency, as we’ve established, is the whole point. But what happens if you actually get hurt?

• You can take a break or train around the injury. For instance, if your arms are injured, you can primarily train your legs or find low-intensity training that lets you keep moving. Back in 2024, I hurt my knee and couldn’t train or do cardio for about 12 weeks. During that time, I relied on daily “hurt knee” exercises that I found on YouTube to train my legs and used a pair of dumbbells to continue training my upper body.

• When you eventually come back from an injury (or if you haven’t trained in a while), start really easy and light, and gradually build back up.

A Note on Aesthetics (and Abs)

Image Source: Glasbergen Cartoon Services

Let’s be real, a lot of people start training because they want to look better. That’s a perfectly valid reason, and there’s no need to pretend otherwise. But it’s worth understanding how aesthetics actually work, because it’s not quite what most people expect.

The biggest factor in how “defined” or “toned” you look isn’t which exercises you do; it’s your body fat percentage. Everyone has abs, for instance. Whether you can see them is almost entirely a function of how much fat is sitting on top of them. You don’t need to do 500 crunches every day to get a six-pack. You need to get lean enough for them to show, and then have enough core muscle underneath to make them pop.

This is why nutrition and strength training together matter more than any single exercise. You can do ab workouts every day and never see definition if your body fat is too high. Our bodies also do not spot reduce fat, so doing a lot of core workouts would not reduce belly fat (despite what social media tutorials might suggest). And you can do zero direct ab work and have visible abs if you’re lean enough and your core gets trained through compound lifts like squats, deadlifts, and overhead presses.

That said, direct core work still has value — and not just for aesthetics, but for stability, posture, and injury prevention. A few sets of hanging leg raises, cable crunches, or planks a few times a week is more than enough for most people.

The same principle applies to the rest of your body. “Toned arms” or “defined shoulders” are the result of having enough muscle in those areas and being lean enough for that muscle to be visible. The formula is always the same: build the muscle through strength training, reveal it through nutrition.

One more thing worth saying: the level of leanness where abs are clearly visible (roughly 10–14% body fat for men, 18–22% for women) is achievable and maintainable for many people, but it does require consistent attention to diet. The ultra-shredded look you see on magazine covers (sub-8% for men, sub-16% for women) is a different story. That’s the territory we talked about earlier, where health starts to suffer. As always, the goal is a look you can sustain while still sleeping well, performing well, and enjoying your life.

What I Do

I want to note that my training regimen is currently on the intense side, but I’ve built up to this level over several years. If you’re new to training, I recommend starting easy and working your way up. I’ve also been training for a race, so my schedule has more running than normal. But for full transparency, here’s what a typical week looks like for me:

Closing: So What is the Actual Point

So, getting lean before GTA 6. Is that a real deadline? Not really. Rockstar could always push it back from its currently scheduled release date of November 19, 2026. But the deadline was always arbitrary. The habits are what matter, and the goal is always to be as healthy as possible.

What I’ve learned is that none of this requires a dramatic transformation or a perfect plan. It just requires a few simple things done consistently: eating in ways we can sustain, training in ways we enjoy, tracking the trend, adjusting when needed, and giving ourselves long enough timeframes to let it work.

So if you’re reading this and feeling like you want to change something about how you treat your health, or perhaps you are just really interested in hot boy or hot girl summer, my recommendation is to pick one thing. Maybe it’s tracking your weight for two weeks. Maybe it’s hitting a protein target. Maybe it’s going for a walk every day. Start there, build the habit, and add from there.

The goal isn’t a number on the scale or a particular date. It’s learning how to care for ourselves in ways that we can keep doing, even as life happens to us. And hey… if we pull this off, maybe by the time GTA 6 drops we’ll be fit enough to sit on the couch for 100 hours straight and finish it. That’s got to count for something; if nothing else, it’s endurance training.

References and Useful Resources

If you only pick 5 (Starter Pack)

• TDEE estimate: Calculator.net TDEE Calculator

• Food tracking: Cronometer

• Workout tracking: Hevy

• Beginner strength plan: r/Fitness Basic Beginner Routine (Fitness Wiki)

• General guide: The Fitness Wiki

Calculators and Tools

• Sailrabbit BMR Calculator

• MyFitnessPal BMR Calculator

• Calculator.net TDEE Calculator

Food and Nutrition Tracking

• Cronometer — detailed food/activity tracking with micronutrient data

• MyFitnessPal — huge food database, great for quick calorie logging

• MacroFactor — adaptive TDEE tracking based on your weight trend (paid)

• Lose It! — simpler alternative with a clean interface

Workout Tracking

• Hevy — popular gym logger with social features + exercise library (free tier)

• Strong — clean, minimal workout tracker for lifting (free tier)

• StrongLifts 5×5 — app built around the StrongLifts beginner program

• Fitbod — AI-generated workouts that adapt to your equipment and recovery

• Strava — tracking for runs, rides, and outdoor cardio

• Nike Training Club — free guided workouts (strength, yoga, HIIT, bodyweight)

Exercise Form and Learning

• Hevy Exercise Library — searchable database with form guides

• Muscle & Strength Exercise Database — video demos organized by muscle group

• ExRx.net — comprehensive exercise encyclopedia

Beginner Strength Programs

• r/Fitness Basic Beginner Routine — simple barbell program to build the habit

• 5/3/1 for Beginners — Jim Wendler’s system adapted for novices

• GZCLP — flexible linear progression program

• r/Fitness Recommended Routines — curated list of programs for all levels

• Nerd Fitness: Strength Training 101 — approachable guide for absolute beginners

Guidelines and Health Authorities

• American Heart Association: Physical Activity Recommendations

• CDC: Adult Activity Guidelines Overview

• WHO: Physical Activity Guidelines

• Physical Activity Guidelines for Americans (PDF)

Evidence-Based Training & Nutrition (Articles / Podcasts)

• Stronger By Science — evidence-based training and nutrition articles

• Stronger By Science Podcast — deep dives on research + practical takeaways

YouTube — Evidence-Based / Educational

• Jeff Nippard — science-backed breakdowns of training/nutrition

• Jeremy Ethier — biomechanics and evidence-based training

• Natacha Océane — HIIT, nutrition science, fitness experiments

• Renaissance Periodization (Dr. Mike Israetel) — hypertrophy + training science

Note: Some channels mix evidence with opinion/marketing. Use them as tools, not gospel.

YouTube — Form / Practical Coaching (often opinionated, still useful)

• ATHLEAN-X (Jeff Cavaliere) — form cues, injury prevention, exercise selection

YouTube — Fitness Commentary and Transparency

• Greg Doucette — blunt fitness advice + industry commentary

• More Plates More Dates (Derek) — PED transparency + fitness/entertainment analysis

• Sean Nalewanyj — no-nonsense lifting + nutrition advice

• Will Tennyson — entertaining food/fitness content with calorie breakdowns

YouTube — Follow-Along Workouts

• Fitness Blender — free full-length workouts (HIIT, strength, low-impact)

• Caroline Girvan — structured strength programs

• Yoga with Adriene — accessible yoga for recovery and stress

• MadFit — short home workouts, often music-driven

Running

• The Run Experience — running form + plans + beginner guidance

• Nick Bare — hybrid training (lifting + endurance), marathon prep, nutrition

Research and Health Resources

• Exposure to idealized physiques and body dissatisfaction (PubMed)

• Anabolic Steroids and Health Risks (NIDA)

• Muscle dysmorphia and social media exposure (PMC)

• RED-S: Relative Energy Deficiency in Sport (PubMed)

• The Fitness Wiki — community-maintained FAQs + routines + nutrition primers

Articles and Interviews (Media / Hollywood Physiques)

• Zac Efron on his Baywatch transformation (Variety)

• Kumail Nanjiani’s Eternals transformation (Vulture)

• Kumail Nanjiani on turning fitness into a full-time job (Business Insider)

• Chris Hemsworth on maintaining a Thor-level physique (Glamour)

• Chris Hemsworth’s shift toward recovery and sustainability (Economic Times)

Documentaries and Media

• Bigger, Stronger, Faster — documentary on PED culture in America

• More Plates More Dates: PED use in fitness and entertainment (YouTube)

Image Source: XKCD

Introduction

When we write code, we create data (values and objects), and that data must be stored somewhere. Most of the time, that “somewhere” is memory managed by the operating system and the language runtime. You can think of it as a big address space where your program stores the bits that make up objects: numbers, strings, arrays, structs, class instances, and so on.

In some (typically lower-level) languages, creating an object often means explicitly allocating memory for it. You have to know how much you need, avoid writing past the space you were given, and, crucially, remember to free that memory when you’re done. If you don’t, the program may continue to accumulate unused allocations, resulting in a memory leak. If you free the wrong thing (or use memory after it’s been freed), you can get nastier failures — crashes, corrupted data, security vulnerabilities, the whole mess. (If you want examples of how memory bugs can spiral into real-world issues, here are recent write-ups involving vulnerabilities in MongoDB: MongoDB Unauthenticated Attacker Sensitive Memory Leak, Merry Christmas Day! Have a MongoDB security incident.)

Image Source: system32comics (Pardon the NSFW reference)

Keeping track of every allocation in a complex program is tedious and error-prone, which is exactly why many modern languages introduced garbage collection. In garbage-collected languages, you allocate memory freely, and the runtime periodically identifies memory that’s no longer reachable by your program and reclaims it for reuse.

That sounds like “taking out the trash.” However, it’s often closer to recycling: identifying what’s still in use, separating it from what isn’t, and reorganizing memory so new objects can be allocated efficiently. In this article, we’ll demystify how garbage collection actually works and perhaps, why “memory recycling” might be the more honest name. We’re going to peek under the hood (or into the bin) and see what’s really going on.

The Heap (of Garbage?)

Before we get into how garbage collection works, it’s helpful to know where it works. Most programs use two main regions of memory, the stack and the heap.

The stack is like a fast, automatic workspace your program uses for things like function calls and local variables. Every time a function runs, it gets a stack frame; when the function returns, that frame is popped off. Memory on the stack is reclaimed immediately and predictably, so it doesn’t need any special cleanup. Stack allocation is “automatic” in the literal sense: entry into and exit from a function determine the lifetime of the memory used by that function call.

Image Source: Fundamentals of Garbage Collection

The heap, on the other hand, is a shared pool of memory used for data that needs to live beyond a single function call, or whose size/lifetime is hard to predict in advance (think objects, strings, arrays, dynamic data structures, and so on). You can allocate heap objects and keep them around by passing references across functions, storing them in other objects, or placing them in long-lived structures.

And that flexibility is exactly why the heap needs help. Heap allocations don’t have an obvious “end of life” the way stack frames do. In languages without a garbage collector, the programmer is responsible for returning heap memory when it’s no longer needed. In garbage-collected languages, the runtime takes on that job by periodically scanning the heap, figuring out what is still reachable, and reclaiming space from objects that aren’t.

What is Garbage?

It is the runtime’s job to determine which heap objects are still in use and which can be reclaimed. But how does it make the distinction? It can’t read the programmer’s intent, so it doesn’t know what they plan to do with an object. What it can do is check whether the program can still reach an object. If an object is still reachable by the running program, it’s alive. If it’s unreachable, it’s garbage and can be cleaned up (or the space reclaimed for use by a different object).

We can think of the program’s memory as a graph:

• Objects are nodes (strings, arrays, class instances, etc.)

• References/pointers are edges between nodes

If you can start from a known “entry point” and follow references to an object, then the program could still access it. That object must be kept.

If there’s no path to an object, then no code can ever get to it again. It might as well not exist, so the runtime is free to reclaim its memory.

The Roots: Where Tracking Begins

Garbage collectors start from a small set of “definitely alive” references called roots. The exact types of roots vary by language and runtime, but typically include:

• Local variables on thread stacks (things currently in scope)

• Global or static variables

• CPU registers

• References held by the runtime itself (e.g., JIT/compiler metadata)

You can think of roots as places the program can directly “grab” data from at any moment.

Tracing the object graph

Image Source: Introduction to Garbage Collection - Grinnell College

From the roots, the GC traces outward:

1. Look at every root reference.

2. Mark the object it points to as alive.

3. Follow that object’s references to other objects.

4. Repeat until there’s nothing new to visit.

At the end of this traversal, the GC has effectively partitioned the heap into two sets:

• Reachable (alive): the program can still get to these objects.

• Unreachable (garbage): nothing points to them anymore, so the memory they occupy can be reclaimed.

A tiny example

Imagine this situation:

• A root points to object X

• X points to Y

• Y points to Z

So the GC can reach X -> Y -> Z, and all three are alive.

Now, suppose the code drops the last reference to X (e.g., a function returns or a variable is overwritten). If X is no longer reachable from any root, then Y and Z become unreachable too, even if they still point to each other.

The garbage collector doesn’t care whether objects reference each other. It only cares whether the program can reach them from the roots. This also enables tracing to handle cycles naturally. Two objects can be locked in an eternal embrace (like the Lovers of Valdarao), but if nothing reachable points to them, the GC can still collect them.

Now that we know what garbage is, let’s look at the different strategies runtimes use to find and collect it.

Strategies for Garbage Collection

There are multiple ways to take out the trash (or run a recycling program). Over the decades, language designers have developed several strategies for finding and reclaiming unreachable objects, and many real-world runtimes use hybrid approaches. Below are some of the most common methods.

Reference Counting: Keeping a Tally

In this method, each object maintains a counter of the number of references pointing to it.

• When we create a new reference to an object, its count goes up.

• When a reference goes away (a variable is overwritten, a scope ends), the count goes down.

• When the count hits zero, the object is immediately collectible, because nothing can reach it anymore.

For instance:

x -> [A] (refCount = 1)
x = null
[A] (refCount = 0) -> reclaim A

The appeal is simplicity and immediacy: objects are cleaned up the moment they are no longer in use, with no need to pause the program for a large collection phase. Languages like Python and PHP use reference counting at the core of their memory management.

The tradeoff here is circular references. If object A points to B and B points back to A, both counters stay at 1 even if nothing else in the program can reach either of them. The memory “leaks” unless the runtime has a secondary mechanism (like Python’s cycle detector) to catch these cases.

There’s also overhead to consider. Every assignment, every function call, and every time a reference is copied or dropped, the runtime must update the counter. In reference-heavy code, the bookkeeping can add up.

Mark-and-Sweep (Tracing GC)

Image Source: M17, CC0, via Wikimedia Commons

Mark-and-sweep takes the reachability approach we described in the previous section and turns it into a two-phase algorithm.

• Mark phase: Starting from the roots, the GC traverses the object graph and marks every reachable object as alive.

• Sweep phase: The GC walks through the entire heap. Any object that wasn’t marked is unreachable, so its memory is reclaimed. Marked objects then have their marks cleared in preparation for the next cycle.

Root -> A -> B

Heap:  [A]   [B]  [C] [D]
Mark:  keep  keep  -   - 
Sweep: reclaim C and D

The big advantage over reference counting is that mark-and-sweep handles cycles naturally, because it only cares whether an object is reachable from a root.

The tradeoff is that a full tracing pass can be disruptive. In a basic mark-and-sweep implementation, the program often pauses while the GC traces and sweeps the heap. The larger the heap, the more work there is to do, and the more noticeable the pause. Modern collectors mitigate this with incremental and concurrent techniques that do more of the work alongside the running program, but the fundamental tension between thoroughness and responsiveness remains.

Generational Collection: Most Objects Die Young

Generational collection is built on an empirical observation—the generational hypothesis: most objects have very short lifetimes. A temporary string built during a function call, an iterator, an intermediate calculation… these objects are created, used briefly, and become garbage almost immediately. A smaller fraction of objects (configuration, caches, long-lived data structures) stick around much longer.

Generational collectors exploit this by dividing the heap into regions, typically called generations:

• Young generation (nursery): Where new objects are allocated. This region is collected frequently because most of its contents are expected to die quickly. Since the region is small, these collections are fast.

• Old generation (tenured): Objects that survive one or more young generation collections get promoted here. This region is larger and is collected less often, since its objects have already proven likely to persist.

New objects -> Young (collected often) -> survivors -> Old (collected rarely)

Image Source: The CS 6120 Course Blog - Cornell University

The tradeoff is that we must track cross-generational references. If an old object points to a young object, the GC must not accidentally collect the young one during a “young-only” collection. So the runtime does bookkeeping (often via a write barrier and a remembered set) to remember “old → young” pointers. That tracking adds overhead, but it’s usually worth it because it avoids frequent scanning of the entire heap.

Many production collectors use some form of generational collection (often combined with concurrent or incremental techniques to reduce pause times). Examples include the JVM’s generational collectors, .NET’s GC, and JavaScript engines like V8.

Real-world collectors rarely use a single strategy in isolation. Modern GCs mix and match: copying and compacting collectors relocate surviving objects to reduce fragmentation; incremental and concurrent collectors interleave GC work with program execution to reduce pause times; and region-based collectors (like Java’s G1) divide the heap into independently collectible chunks and prioritize the ones with the most garbage. Some runtimes also try to avoid GC work entirely when possible via escape analysis; they prove that certain objects don’t outlive a function and allocate them on the stack instead, so the collector never has to touch them.

At the far end of the spectrum is the “no GC at runtime” approach: languages like Rust enforce ownership rules at compile time and free memory deterministically, without a runtime collector. The takeaway is that garbage collection isn’t a single algorithm. It’s a toolbox, and most production runtimes are creative hybrids tuned for different tradeoffs.

The Cost of Recycling

Garbage collection saves the programmer from having to manage memory themselves, but it doesn’t make memory management free (even though we never have to call free()). It shifts the cost from the code to the runtime, and that cost can show up in a few ways.

Pause Times

This is the most visible cost. When the GC runs, it may need to pause your program—or at least slow it down. In a basic stop-the-world collector, everything freezes while the GC traces and sweeps. These pauses can range from microseconds to tens of milliseconds (and, in extreme cases, longer), depending on heap size, the number of live objects, and the collector’s design.

For a batch job or a CLI tool, you might never notice. For a game rendering at 60 FPS, a multiplayer server, or a trading system, even a few milliseconds in the wrong place can be a real problem. This is why so much GC research focuses on reducing, hiding, or distributing pause times. Concurrent collectors, incremental passes, and region-based collection exist largely to address this.

Throughput Overhead

Even when the GC isn’t pausing your program, it’s still consuming resources. Tracing the object graph, maintaining write barriers for generational collectors, and copying or compacting live objects all consume CPU time that could otherwise be spent on your application logic. (And in runtimes that use reference counting/ARC, reference updates add overhead too.)

The overhead varies widely. A well-tuned collector might use only a small percentage of total CPU time. A poorly tuned one—or one dealing with a pathological allocation pattern—can eat significantly more. The general rule is that GC trades some raw throughput for the convenience (and safety) of not managing memory yourself.

Memory Overhead

Garbage collectors need headroom to work efficiently. If the heap is nearly full, the GC has to run more frequently, which increases pause times and CPU usage. Most collectors perform best when the heap has some breathing room, which means your program may use more memory than it strictly needs at any given moment.

Some strategies incur additional memory costs on top of this. Copying collectors need space to copy live objects into. Generational collectors maintain remembered sets to track cross-generational references. Reference counting adds metadata to track counts. None of these is enormous individually, but they add up, especially in memory-constrained environments, and especially when fragmentation forces the runtime to keep extra slack space around.

Unpredictability

Perhaps the subtlest cost is timing: you don’t control exactly when the GC runs, and you don’t control exactly how long it takes. You can influence it through tuning and allocation patterns, but ultimately, the runtime decides when to collect and how much work to do. That can make GC-related performance issues harder to reproduce, profile, and reason about than many other bottlenecks.

In manually managed languages, freeing memory is explicit and predictable: you pay the cost exactly where you wrote the free() call. With GC, that cost is deferred and amortized in ways that can be hard to connect back to the allocation behavior that triggered it.

So, is it worth it?

For most applications, yes—overwhelmingly. The bugs that manual memory management introduces (use-after-free, double-free, leaks) are often far more expensive in practice than the overhead a modern GC imposes. But “most applications” isn’t all applications, and understanding what you’re trading away helps you make informed decisions about when GC is the right fit and when you might want more control.

Conclusion: Better Than Recycling

We’ve been calling garbage collection a recycling program, and the metaphor holds up surprisingly well. The runtime traces what’s still reachable, separates it from what isn’t, and reclaims the leftovers so they can be put to work again.

But here’s where the metaphor breaks down—and in the best possible way. Real-world recycling degrades material. Paper loses fiber. Plastic weakens. Every cycle produces something a little worse than what came before. Reclaimed memory has no such problem. A freed byte is identical to a fresh one. There’s no wear, no downgrade, no loss of quality. The runtime hands it back in perfect condition, ready to become whatever your program needs next.

So the next time your program hums along without you writing a single malloc() or free(), take a moment to appreciate the quiet, efficient system working behind the scenes. It’s not just taking out the trash; it’s running a recycling program that never loses fidelity. And it’s doing it while you’re not looking.

Working on this blog post has been a lovely learning experience, and I thought it would be fun to write a small garbage collection library for C to harden the concepts. If you are interested in learning along, feel free to look at the repo and contribute. I put together a roadmap (with some help from Claude) and will walk through it and learn as we go. Find the repo here: garbagec.

References

• Incremental Garbage Collection in Ruby 2.2 — Heroku

• Baby’s First Garbage Collector — Bob Nystrom

• The Garbage Collection Handbook — gchandbook.org

• Boehm-Demers-Weiser GC — Hans Boehm

• Crafting Interpreters, Ch. 26: Garbage Collection — Bob Nystrom

• Writing a Simple Garbage Collector in C — Matt Plant- What different types of garbage collection are there? — Language Dev Stack Exchange

• Garbage Collection — Grinnell College CS302

• Fundamentals of Garbage Collection — Telerik

• Back to Basics: Generational Garbage Collection — Microsoft

• Java Garbage Collection Overview — Oracle

• Teaching Garbage Collection — Brown PLT

• Incremental GC — Heroku

• A Unified Theory of Garbage Collection — Cornell CS6120

• An Animated Guide to How Garbage Collection Algorithms Work — freeCodeCamp

• Memory Management Strategies: Garbage Collection, Reference Counting, and Beyond — Andrew Johnson

• Garbage Collection — Cornell CS4120

• Garbage Collection in Python — GeeksforGeeks

• Reference Counts — Python Docs (Extending Python)

• How does Python’s garbage collector detect circular references? — Stack Overflow

• Reference Counting — Python C API Docs

• Deep Understanding of Garbage Collection: Principles, Algorithms, and Optimization Strategies — threehappyer

• Generational Garbage Collection and Heap Memory — Incus Data

• Generational GC — Northwestern CS321

• MongoBleed Detector — Neo23x0 (GitHub)

• Fundamentals of Garbage Collection — Microsoft .NET Docs

• Introduction to Garbage Collection — dev.java

• Garbage Collection in Java — IBM

• How Does Garbage Collection Work in Java — Alibaba Tech

• Garbage Collection — ODU CS330

• Garbage Collection Algorithm: Mark-Sweep — Animesh Gaitonde

• Mark and Sweep Garbage Collection Algorithm — GeeksforGeeks

• Understanding Garbage Collection — Aerospike

• Understanding Garbage Collection — Naik

• Garbage Collection in Java — GeeksforGeeks

“From the earliest times, salt has been a symbol of lasting alliances, sacred rituals, and even rebellion.” — Mark Kurlansky, Salt: A World History

Salt once shaped empires, sparked wars, and preserved entire civilizations. It helped our ancestors store food during long journeys and harsh winters. In Salt: A World History, Mark Kurlansky traces the remarkable influence of this humble mineral, describing how it carved trade routes, powered economies, and quietly shaped the course of history.

Salt still preserves in our modern, digital world—not food this time, but something just as vital: our online identities. Without salting, even the most complex passwords can be cracked in minutes, leaving personal data exposed and vulnerable.

In this post, we’ll explore how a simple pinch of salt can disrupt even the most sophisticated attacks. If that sounds mysterious or technical, don’t worry — we’ll break it down gently. Think of it like a recipe: no dish is complete without a bit of salt, and no password system is secure without it, either.

Image Source: XKCD

The Problem with Passwords

Passwords are one of the oldest tools we have for identification and authentication. If we trace their history, we can go all the way back to the Roman army, where soldiers used watchwords to prove they belonged to a particular unit. Fast forward a bit, and we get the secret codes whispered at speakeasies during Prohibition — the price of admission for a sip of forbidden liquor. The first digital password was introduced at MIT in 1961 by Fernando J. Corbató, a pioneering computer scientist who wanted to increase access to shared machines while keeping users’ data private.

Passwords have stuck around since then, but they’ve also become one of the weakest links in our digital defenses. On a human level, most of us reuse the same password across multiple accounts or choose ones that are far too easy to guess (password123, iloveyou, etc.). Even when we try to be clever, the truth is — humans aren’t great at randomness. And hackers know that.

Worse still, some systems continue to store passwords in plain text. Yes, in plain text — as recently as 2019, Meta (the parent company of Facebook) discovered it had stored over 600 million user passwords in searchable plain text files dating back years. In 2021, a GoDaddy breach exposed the plain-text passwords of 1.2 million users. Storing passwords in plain text is exactly as bad as it sounds. It’s like writing down your deepest secrets in a diary and leaving it unlocked on a public bench. If anyone gains access, they can read everything. To fix that, developers use hashing, which turns passwords into scrambled, fixed-length codes that are difficult to reverse. For example, instead of storing this:

We use a hashing function to transform it into this:

This way, even if an attacker gets the data, they see only hashes — not the actual passwords. When a user logs in, the password they enter is hashed, and the system compares it to the stored hash.

But hashing isn’t bulletproof. If two users choose the same password, their hashes will be identical — making it easier for attackers to spot patterns. Hackers also use rainbow tables , which are massive precomputed lists of common passwords and their corresponding hashes. And brute-force attacks like dictionary attacks can try billions of combinations per second. The more predictable the system — or the hashing function — the easier it is for attackers to win.

So how do we throw them off the scent? How do we make even predictable passwords harder to crack?

That’s where salt comes in.

So... What is a Salt?

In the world of password security, a salt is a random string of characters added to a password before it’s hashed. It doesn’t have to be long or fancy — though a long enough salt almost eliminates the risk of a rainbow table attack. This tiny addition completely changes the outcome of the hash, even if two users pick the exact same password.

Think of it like a secret spice in a recipe. Two chefs might start with the same dish, but if one adds cinnamon and the other adds chili, you’ll end up with very different flavors. That’s what salt does: it makes every “dish” — in this case, every password hash — unique.

Let’s break it down with an example.

Suppose two users both choose the same (very strong!) password: password.

If we hash it without any salt, they’ll get the exact same result:

Hash("password") → $2y$10$LYkedKLZX3qDb06Hd17zWuybj9EhqUWKFjOjqSRBKF6q3rE3ldcsq

Now, let’s say we add a unique salt for each user:

User1: Salt = X8g72!
Hash("X8g72!password") → $2y$10$DebUSiinkd2GvJZ0uqgsFOvvfDClW.kvnZ7R3g/DLmMh/XySy00hu

User2: Salt = Lz91#k
Hash("Lz91#kpassword") → $2y$10$mIzd3Z5y.dwyTVD.EpCeV.vZOp8dPUSzY3SRt.Whrj.o8kDAIDhaq

Even though both users chose the same password, their final hashes are completely different. That means a hacker can’t easily tell they started from the same input. And because salts are generated randomly, rainbow tables become practically useless — an attacker would have to compute a new table for every possible salt, which is computationally impractical.

Using the earlier example from the hashing section, we can now see how salting changes what’s stored in the database. First, here’s how we build the salted password:

And here’s how we safely store both the salt and the resulting hash:

By storing the salt alongside the hash, systems can reliably re-hash a password during login for comparison without compromising security. The salt itself doesn’t need to be secret. It just needs to be unique and unpredictable.

It’s important to emphasize, however, that salting doesn’t make passwords stronger. It only makes attacks harder — and that distinction matters.

Common Mistakes

Image Source: XKCD

"You are the salt of the earth. But if the salt loses its saltiness, how can it be made salty again? It is no longer good for anything, except to be thrown out and trampled underfoot." - Jesus, Matthew 5:13

Like any good recipe, salting only works if you do it right. One of the biggest mistakes is reusing the same salt for every user — which completely defeats the purpose. If everyone gets the same seasoning, all the hashes for identical passwords will still look the same. That’s no better than unsalted hashing. Salts must be unique per user — and ideally unique per password reset — to offer real protection.

Another common pitfall is using salts that are too short, predictable, or not truly random (we can talk about generating secure random values in a follow-up post). A salt like 123456 or admin isn’t fooling anyone — certainly not a hacker with a rainbow table and some spare time. Ideally, a good salt should come from a strong random number generator, be at least 16 characters long, and draw from a broad character set. You can dig deeper in this Stack Overflow conversation on optimal salt lengths.

Other frequent missteps include forgetting to store the salt or storing it insecurely. While the salt doesn’t need to be hidden, it does need to be reliably retrievable. If you lose or corrupt it, you cannot re-hash and verify a user’s password correctly—it’s like losing the recipe for your signature dish.

And finally, one of the most unfortunate (and surprisingly common) mistakes is hashing without salting. (You can find examples in the reference links below.) This still shows up in legacy systems or rushed codebases — and it leaves users dangerously exposed to the kinds of attacks we’ve already discussed. Modern systems should always salt and hash passwords with a strong algorithm like Argon2, or bcrypt. Even though limitations in bcrypt contributed to the recent Okta security vulnerability, it’s still widely used — and I have some thoughts on Okta Security coming soon, so stay tuned).

For practical implementation tips, I also highly recommend the OWASP Password Storage Cheat Sheet.

Bonus: Salt Alone Isn't Enough (Use Pepper too)

Salt is powerful — but it’s not a magic shield. While it protects against attacks like rainbow tables and ensures every password hash is unique, it’s just one part of a broader security recipe.

Some systems add another secret ingredient: a pepper. Like salt, a pepper is a string added to the password before hashing — but unlike salt, the pepper is kept secret and shared across all users. Think of it as a house spice blend only the kitchen staff knows. Even if attackers access the database, they still need the pepper to recreate the hashes.

That said, pepper comes with tradeoffs. Because it must be stored securely (usually in the application code or an environment variable), managing it introduces additional complexity. As with any ingredient, there’s ongoing debate—some security experts argue that pepper doesn’t always add meaningful protection and may not be worth the added risk.

Then there’s the matter of how we hash. Not all algorithms are created equal. General-purpose functions like SHA-256 are fast — which is excellent for performance, but unfortunately, it’s also great for attackers. That’s why we use key-stretching algorithms like bcrypt, scrypt, or Argon2, which are intentionally slow. These algorithms increase the cost of each password guess, making brute-force and dictionary attacks much less practical.

And, of course, password storage is just one layer of defense. A secure system should also include rate limiting (to prevent automated guessing or bot-like behavior), multi-factor authentication (MFA) (so a stolen password alone isn’t enough), and regular audits of how passwords and credentials are managed.

Security is layered. Salt is essential, but it can’t carry the weight alone. It works best when combined with solid algorithms, thoughtful design, and strong overall hygiene. Like seasoning a meal, security is not just about one ingredient—balance, technique, and attention to detail.

Conclusion: Security Recipe Card

As a bonus, I created the hashes in the examples above using one of the common key-stretching algorithms. If you can figure out which one, let me know. I’d be curious to learn how you did it.

Salting might seem like a quirky detail in the vast world of cybersecurity, but as we’ve seen, it’s one of the simplest and most powerful tools we have for protecting passwords. By adding randomness to each hash, salts break patterns, frustrate attackers, and help ensure that even identical passwords don’t leave identical traces.

For developers, salting should be a non-negotiable part of your password storage process. And for the curious reader — whether you’re learning, building, or just staying security-conscious — it’s worth digging into how your favorite platforms handle password protection. Do they use proper salting? Do they mention bcrypt or Argon2? Is MFA available? These are questions worth asking.

Password security doesn’t have to be complicated — but it does have to be thoughtful. So yes: salt your food, and your passwords.

Finally, while this post focused on password security, my next post will take a deeper look at digital identity — and how it shapes, protects, and sometimes complicates our lives online. Hope to see you there.

Till then, stay salty, stay safe. 🧂😌

References and Resources to Learn More

• Hackers Stole Access Tokens from Okta’s Support Unit

• Unpacking the Okta Data Breach

• Is salted MD5 or salted SHA considered secure?

• Why an unsalted MD5 hash is bad practice

• How NOT to Store Passwords! - Computerphile

• ELI5: What does it mean if a password is stored as unsalted MD5 hash?

• What does it mean if a password is stored as an unsalted MD5 hash?

• The Evolution of Password Hashing

• Legacy app with md5 hashes: how to add salt and SHA1?

• What is bcrypt and how does it work?

• Okta AD/LDAP Delegated Authentication - Username Above 52 Characters Security Advisory - Nov 1, 2024

• How Bcrypt’s Limitations Contributed to Okta’s Vulnerability: A Lesson for Developers

• Is it an good idea of Using password itself as an salt

• Best Practices: Salting & peppering passwords?

• Password Hashing and Storage Basics

• Password Salting

• Interesting Conversation from Y Combinator News

• Adding Salt to Hashing: A Better Way to Store Passwords

• What is a rainbow table attack and how to prevent it?

• Salt – Preventing Rainbow Attacks against Password Stores

• Salt Hashing and How It Can Lower The Chances of Exposing Login Credentials

• The History and Future of Passwords

• The World's First Computer Password? It Was Useless Too

• (Part I of III) The History of Passwords

• The Secret Password to Enter the Speakeasy

• Meta stored 600 million Facebook and Instagram passwords in plain text

• A Brief History of Passwords

• Facebook Stored Hundreds of Millions of User Passwords in Plain Text for Years

Image Source: XKCD

Introduction: Touchdown?

“Touchdown!” I remember shouting with everyone else during my first homecoming in college as I watched Dartmouth demolish Yale in a 42-10 win. I didn’t fully understand the sport. Still, I knew that when a player in green managed to get the ball into the Yale end zone, it was a touchdown.

At the time, I figured that while I didn’t quite grasp the rules, I enjoyed all the fanfare surrounding American football (I’ll mostly call it “football” from now on for ease—I know, I know, the Nigerian in me is resisting this). The energy was infectious, and I wanted to learn how to play. I casually joked about it over the phone with my mom. Naturally, she showed great concern, reminding me to focus on the real reason I went to school and not get caught up in a sport as dangerous as football.

Speaking of interesting encounters between Africans like me and their mums, my friend Chukwuka recently wrote a brilliant article about “African Mom Jokes” and how they can sometimes go too far, reinforcing harmful stereotypes and distorting reality. In his piece, he calls for more nuanced and respectful portrayals of African mums in media, especially in short-form comedy videos. It’s brilliant writing, and you can read it here.

My curiosity about football faded as schoolwork piled up, forcing me to focus and abandon my pipe dream of a football career. So, as you might have guessed, I never thoroughly learned the sport. But a few months ago, a friend and mentor at work asked if I’d like to join her flag football team. She knew I could run and figured I’d pick up the rest of the skills. I accepted. I can finally say, Mama, I made it—even if I play a much less intense version of the sport. You can read about my adventures during my first season in my 2024 reflection blog post.

My first game? All I had to do was run and chase people—that part was easy. But I was still pretty clueless. So, over the next week, I read several articles, watched countless videos, and learned a lot about football—from its history to the leagues, the teams, the legends, and even the NFL’s “socialist” revenue-sharing model.

As we approach Super Bowl LIX, I know you might be interested in many random things. If you’re a Swiftie, you might only be looking forward to seeing her dance and cheer on her boyfriend. Maybe you’re hoping for an electrifying halftime performance from Kendrick Lamar, given his incredible year and recent Grammy wins. Or perhaps you want to hang out with friends, bring beer (or not—mummy, I know you’re reading this), and watch the ads.

But if you’re interested in the game itself—how it works, how teams score, what the rules are—this blog post should help. After reading it, I suspect you won’t be as lost as I was at the many football games I attended in college. Keep in mind that the focus is exclusively on the game. Ultimately, I will link plenty of resources to learn more about other fascinating aspects of football.

And if you happen to be in Seattle and want to watch the Super Bowl with a friendly group, I’m hosting a watch party—feel free to contact me for more details.

The Basics of American Football

Image Source: Cartoon Stock

American football is one of those sports that looks like absolute chaos until you understand the method behind the madness. Big guys in pads crash into each other, the game stops every few seconds, and yet, millons of people swear by it. If you’re here, you’ve probably wondered: What is happening? Well, at its core, football is a strategic battle for territory, where teams fight (not literally, though sometimes it feels like it) to advance the ball down the field and score points. Once you grasp the fundamentals, though, everything- even the weird rules- starts making sense.

The Goal: Move the Ball, Score Points

The entire point of football is to get the ball (or egg, as some of my friends call it—it’s actually an oblong sphere called “The Duke.” All official balls are handcrafted by Wilson Sporting Goods) into the opponent’s end zone and score more points than the other team. The offense (the team with the ball) has four tries or plays—called downs—to move at least 10 yards. If they succeed, they reset to another four downs and keep advancing. If they fail, they either punt the ball (kick it downfield to push the opponent farther from their end zone) or go for it on fourth down and risk turning it over. Meanwhile, the defense is there to disrupt, tackle, and take the ball back.

The points come in a few flavors:

• Touchdown (6 points): Run or catch the ball in the opponent’s end zone.

• Extra Point (1 or 2 points): After a touchdown, the team can kick for 1 point or run a short play to get the ball into the opponent’s end zone for 2 points.

• Field Goal (3 points): Sometimes, a team gets close to the end zone but can’t score a touchdown—maybe the defense is too strong, or there’s not enough time left for more plays. In these situations, the team can kick the ball through the uprights for three points rather than risking a turnover on fourth down.

• Safety (2 points): Rare but exciting, this occurs when the defensive team forces the offense into their own end zone. This can happen in a few ways:

  • The offensive ball carrier is tackled in their own end zone.

  • A fumble or bad snap causes the ball to roll out of the back of the offense’s end zone.

  • The quarterback commits intentional grounding in the end zone (throws the ball away under pressure with no receiver nearby).

  • A blocked punt or kick bounces back into the end zone and goes out of bounds.

  When a safety occurs, the defense is awarded 2 points, and the team that gave up the safety must kick the ball away from their 20-yard line, handing possession to the opponent.

The Field: A 100-Yard Chessboard

Image Source: How Stuff Works

The game is played on a 100-yard field (plus two 10-yard end zones). Every 10 yards is marked with bold white numbers - 10, 20, 30, all the way to the 50-yard line at the center of the pitch. In the metric system, 10 yards is about 9.14 meters. The ball’s position dictates where the next play starts, and there are lighter 5-yard lines and tiny white hash marks to ensure precise placement. Finally, the big yellow goalposts are where kickers (more on them later) become heroes or villains.

The Game Flow

Football is a game of constant shifts between offense, defense, and special teams:

• Offense: The unit trying to score, led by the quarterback (think of them as the team’s general).

• Defense: The squad trying to stop the offense—tackling, intercepting passes, and causing chaos.

• Special Teams: The often-overlooked unit handling kickoffs, punts, and field goals.

Every game consists of four quarters (15 minutes each) plus halftime. It starts with a kickoff, and from there, teams trade possessions, moving the ball by either running (carrying the ball) or passing (throwing it downfield).

The offense loses possession when there’s a turnover, which can happen in a few ways:

• Interceptions: When a defensive player catches the quarterback’s pass.

• Fumbles: When the ball carrier loses control, the defense recovers.

• Turnover on Downs: When a team fails to gain 10 yards and doesn’t punt, handing the ball to the opponent.

Teams on defense can also gain an advantage through sacks, which means tackling the quarterback behind the line of scrimmage before they can throw the ball. Sacks can cause a loss of yardage, kill offensive momentum, or even lead to a turnover.

Penalties and Game Management

Referees throw yellow flags for rule violations, pushing teams forward or backward. Some common penalties include:

• Offsides and False Starts: Jumping ahead before the play begins.

• Holding: Illegally grabbing an opponent.

• Pass Interference: Preventing a receiver from catching the ball.

• Personal Fouls: Late hits or unnecessary roughness (yes, that’s a thing in football).

Teams strategically use timeouts, the two-minute warning, and clock management to control the pace of the game. If time runs out and the score is tied? Overtime. And trust me, the NFL has robust rules for handling overtime, which you can read about here.

Now that we understand the basics, let’s examine positions and player roles—because knowing who does what is just as important as understanding how the game flows.

Positions and Roles

Image Source: The Shakerite

As we’ve learned initially, teams play offense or defense in football. Similarly, the positions and roles are divided into offensive and defensive positions, each contributing distinctly to helping their team gain an advantage on the field.

Offensive Positions

The offense’s primary objective is to advance the ball down the field and score points. The 11 offensive players are divided into two main groups: the offensive line and the backs and receivers.

Offensive Line

The offensive line (O-line) consists of five players whose primary responsibility is to protect the quarterback and create lanes for the running back.

• Center (C): Snaps the ball to the quarterback at the start of a play and immediately gets to work blocking defenders. Often, the center is the brain of the O-line, calling out adjustments to plays based on the defense’s setup.

• Guards (LG/RG): Positioned on either side of the center, they block defensive players, opening holes for the running back and keeping pass rushers away.

• Tackles (LT/RT): Lining up outside the guards, tackles protect the edges. The left tackle is crucial because they guard the quarterback’s blind side (think The Blind Side movie). They must be quick on their feet, strong, and highly skilled to hold off elite pass rushers.

Backs and Receivers

These players are responsible for advancing the ball through running and passing plays.

• Quarterback (QB): The team’s leader, the play-caller, and often the player that gets all the credit (and all the blame) after plays. The quarterback receives the ball from the center and can pass, handoff, or run the ball. They read the defense and may change plays at the line of scrimmage, a process known as calling an audible.

• Running Backs (RB): Typically positioned behind the quarterback, running backs primarily carry the ball on running plays but can also catch passes. The main types are:

  • Halfback (HB): Usually the primary ball carrier, known for speed and agility.

  • Fullback (FB): Generally larger and stronger, fullbacks lead block for the halfback and occasionally carry the ball in short-yardage situations.

• Wide Receivers (WR): These players line up near the sidelines and are the primary targets for passing plays. They run specific routes to get open for passes and may also block running plays.

• Tight End (TE) (Fun fact: This is the position that Taylor Swift’s boyfriend, Travis Kelce, plays, and that the popular sports media personality and NFL Hall of Fame inductee, Shannon Sharpe, played). Positioned next to the offensive line, tight ends serve dual roles as blockers and pass-catchers. Their versatility makes them valuable in both the running and passing games.

Defensive Positions

The defense’s primary objective is to prevent the offense from scoring by tackling ball carriers, defending against passes, and creating turnovers.

Defensive Line

These players line up directly on the line of scrimmage and engage the offensive line. They are the first line of defense.

• Defensive Tackles (DT): Located at the center of the defensive line, their primary goal is to stop running plays up the middle and pressure the quarterback on passing plays.

• Defensive Ends (DE): Positioned on the edges of the defensive line, they are usually quick and strong and attempt to contain plays by rushing towards the quarterback (sacks, baby!) and stopping outside runs.

Linebackers

Linebackers stand behind the defensive line and are versatile players involved in both run defense and pass coverage.

• Middle Linebacker (MLB): Often considered the “quarterback of the defense,” the MLB directs defensive alignments. It is heavily involved in stopping the run and covering short to intermediate pass routes.

• Outside Linebackers (OLB): Depending on the defensive scheme, OLBs may focus on covering receivers, supporting run defense, or blitzing the quarterback.

Defensive Backs

Also known as the secondary, these players focus on defending against passing plays and providing support on running plays.

• Cornerbacks (CB): Typically covering the offense’s wide receivers, cornerbacks aim to prevent successful passes by deflecting or intercepting the ball.

• Safeties: Positioned deeper in the defensive backfield, safeties provide a last line of defense against deep passes and runs.

  • Free Safety (FS): The last line of defense, often patrolling deep downfield to prevent big plays.

  • Strong Safety (SS): A hybrid linebacker-safety who helps cover tight ends and running backs in passing situations.

Special Teams

Special teams units handle all kicking plays, including kickoffs, punts, field goals, and extra points. Players on special teams have specialized roles that are crucial in determining field position and scoring opportunities.

• Kicker (K): Responsible for kickoffs, field goals, and extra point attempts.

• Punter (P): Handles punts, aiming to pin the opposing team deep in their own territory.

• Long Snapper (LS): Specializes in snapping the ball over longer distances, typically for punts and field goal attempts.

• Kick Returner (KR) and Punt Returner (PR): Tasked with catching kickoffs and punts, respectively, and advancing the ball as far as possible to set up a favorable field position.

For a visual explanation of these positions and their roles, you might find this video helpful: SPORTS 101 // Guide to American Football Positions.

Strategies and Tactics

Image Source: XKCD

Football isn’t just about brute strength and speed; it’s a complex chess match where strategy and tactics play pivotal roles. Coaches and players meticulously plan and execute plays to outmaneuver their opponents, making the game as intellectually engaging as it is physically demanding. Let’s delve into the strategic elements that define this sport.

Offensive Strategies

The offense aims to advance the ball and score points, employing various strategies to keep the defense off balance.

1. Play Calling and Game Planning

Before each game, teams develop a game plan to exploit the opponent’s weaknesses and leverage their own strengths. This plan includes a selection of plays designed for various situations, such as short-yardage gains, long-distance passes, or red-zone scoring opportunities. Coaches script the opening plays to set the tone and gather information on the defense’s tendencies.

2. Formations and Personnel Packages

Offensive formations dictate the alignment of players on the field. Common formations include:

• I-Formation: Features a fullback and a halfback lined up directly behind the quarterback, ideal for powering running plays.

• Shotgun Formation: This formation positions the quarterback several yards behind the center, providing a better vantage point for passing plays.

• Singleback Formation: Utilizes a lone running back, allowing for a balanced attack with multiple receiving options.

Personnel packages refer to the combination of running backs, tight ends, and wide receivers on the field. For example, a “21” personnel package includes two running backs and one tight end. Varying these packages keeps the defense guessing and can create mismatches.

3. Passing vs. Running Plays

Balancing passing and running plays is crucial. A strong running game can force the defense to commit more players to the line of scrimmage, opening up opportunities for play-action passes. Conversely, an effective passing attack can spread the defense, creating running lanes.

4. Tempo and No-Huddle Offense

Altering the tempo of play can disrupt the defense’s rhythm. A no-huddle offense, where the team quickly lines up and executes plays without huddling, can tire defenders and limit their ability to substitute players, potentially exploiting mismatches.

Defensive Strategies

The defense aims to prevent the offense from advancing the ball and scoring, using various schemes and tactics to counter offensive plays.

1. Defensive Alignments and Formations

Defensive formations are designed to counter specific offensive strategies. Common alignments include:

• 4-3 Defense: Four defensive linemen and three linebackers, providing a balanced approach to stopping both the run and the pass.

• 3-4 Defense: Three defensive linemen and four linebackers, offering greater flexibility in blitz packages and pass coverage.

2. Man-to-Man vs. Zone Coverage

In pass defense, teams employ:

• Man-to-Man Coverage: Each defender is responsible for covering a specific offensive player, allowing for tight coverage but susceptible to pick plays.

• Zone Coverage: Defenders cover designated areas of the field, which can be effective against passing attacks but may leave gaps if not executed properly.

3. Blitzing

A blitz involves sending additional defenders, such as linebackers or defensive backs, to pressure the quarterback. While this can lead to sacks or hurried throws, it also leaves fewer players in coverage, increasing the risk of big plays by the offense.

4. Turnover Creation

Defenses aim to create turnovers through interceptions and forced fumbles. Strategies include disguising coverages to confuse the quarterback and employing aggressive tackling techniques to dislodge the ball.

Special Teams Strategies

1. Kickoff and Punt Coverage: The objective is to limit the opposing team's return yardage. This involves strategic kick placement and disciplined coverage lanes to corral the returner.

2. Return Strategies: Return teams design blocking schemes to create lanes for the returner, aiming to gain an advantageous field position or score. Decisions on whether to attempt a return or signal for a fair catch are based on factors like hang time and coverage speed. Keep in mind, however, that the NFL regularly changes the rules of the game to keep things fresh and interesting. Hence, this information might be outdated, especially given the recent change to the kickoff rules.

3. Field Goal and Punt Fakes: Occasionally, teams will attempt fake field goals or punts to catch the defense off guard, aiming to convert a first down or score. These high-risk, high-reward plays are typically used in critical situations.

In-Game Adjustments

Football is dynamic, and successful teams make real-time adjustments based on the game’s flow.

1. Halftime Adjustments: Coaches analyze first-half performance, identifying what’s working and what’s not. They may alter their game plan, adjust play-calling tendencies, or implement new strategies to exploit observed weaknesses in the opponent.

2. Audibles: Quarterbacks have the authority to change the play at the line of scrimmage based on the defensive alignment. This ability to “call an audible” allows the offense to adapt to unexpected defensive strategies.

3. Clock Management: Managing the game clock is a critical strategic element, especially in close games. Teams may use timeouts, hurry-up offenses, or deliberate slow play to control the clock.

Understanding these strategies provides a deeper appreciation for the complexity and excitement of American football. It’s a game where every decision can influence the outcome, making it a thrilling spectacle for fans and a challenging endeavor for players and coaches alike. To understand more of the strategy details you might find these videos and resources helpful: Football Plays, Positions, Strategy & Tactics for Beginners, Smart Football, Breakdowns by Brett Kollmann.

Conclusion

Image Source: XKCD

We’ve covered a lot—from a bit about football rules to the positions, strategies, and tactics that make it all make sense. If your brain is feeling overloaded, that’s normal. Football is a live-action strategy game with 300-pound linemen and commercial breaks every five minutes.

The next time you watch a game, you’ll know why everyone yells at the TV. You’ll recognize when a quarterback calls an audible, when a defense is setting up for a blitz, and when a kicker is probably about to ruin someone’s night. Maybe you’ll even impress your friends by casually explaining what a Cover 2 defense is (make sure to say it with confidence).

And if none of this has convinced you that football is fantastic, well—there are still the halftime show, the snacks, and the ads, so you’ve got something to look forward to.

Now, grab your wings, find a comfy seat, and enjoy the Super Bowl madness or just the halftime show. And if you still don’t get what’s happening? Just cheer when everyone else does. Trust me, it works. 🏈😂

References and Resources to Learn More

• What’s Actually Going On Out There? A Beginner’s Guide to American Football

• A Beginner’s Guide to American Football (NFL); rules, regulations, divisions & more

• NFL Throwback: How EVERY Team Got Its Name & Identity!

• The Entire History of the NFL

• How the NFL used ‘socialism’ to get rich

• The ENTIRE History of the Super Bowl!

• The OFFICIAL History of NFL Rules!

• Rules Of American Football EXPLAINED FOR BEGINNERS

• Rookie’s Guide

• SPORTS 101 // Guide to American Football

• Learn American Football in 5 Minutes

• What are the rules of American football? How does NFL work?

• NFL Rulebook

• [Quick Guide] What are the rules of football?

• A Beginner’s Guide to American Football

• Understanding American Football and Soccer

• Offensive and defensive football positions explained: What are they and why are there two different teams

• Football (American)

• How to Understand NFL Lingo, Part 1 of 3

• How to Understand NFL Lingo, Part 2 of 3

• How to Understand NFL Lingo, Part 3 of 3

2024 New Year Fireworks: Seattle Center

Introduction

I’ll admit it—I’m not the best at telling my own story. While I thrive at spotlighting the work of others, sharing my own journey often feels clumsy or incomplete. But as 2024 ends, I’m challenging myself to pause, reflect, and celebrate the big and small moments that defined my year.

Join me as I recount the highlights, setbacks, and lessons that made 2024 unforgettable. In the paragraphs ahead, I’ll take you through my adventures in sports and physical activity, professional milestones, relationships, community-building, and a few personal favorites that shaped my year. Finally, I’ll share what I look forward to as we enter 2025. Let’s dive into the year that was!

Physical Activity: Lifting Weights, Chasing Wins, and The Art of Running in the Rain

On a chilly, grey January 1st, 2024, I laced up my running shoes and stepped out into the Seattle mist for a one-mile run. My goal for the year was simple: run one mile every day (except Saturdays, with a 5K on Sundays) and hit or surpass 366 miles by year’s end. If I stuck with it, I figured I’d build the strength and consistency to start training for a half marathon by 2026.

So, how did it go? Well, I didn’t quite make 366 miles. Instead, I logged just over 300—thanks to a 12-week break after a stress fracture in my knee from soccer. But here’s the surprising part: despite the setback, I completed a grueling half marathon. And not just any race—it was a NorthWest Trail Run, complete with relentless rain, slick mud, freezing cold, and punishing elevation changes. With no proper training, I finished at a 10:30/mi pace, running straight through and stopping only four times for quick energy gels. It wasn’t pretty, but I embraced the challenge and loved every bit of it. Beyond that, I hit a few personal running milestones that I’m proud of: a best 5K time of 28:08 and my fastest average pace of 7:14/mi over a two-mile run.

Running a Half Marathon

What made me happiest was seeing some of my friends—who weren’t big on running—join me. A few had teased me about “chasing nothing,” so watching them embrace the joys and inevitable aches of running was incredibly rewarding. I couldn’t be more pleased—they finally get it. 😌

Running wasn’t the only activity that kept me moving this year. Soccer played a big role, too. I joined a Seattle Recreational Adult League team in late 2023, holding down the center forward position throughout 2024. Watching our team grow from underdogs to contenders by December was exhilarating. I also joined a flag football team for the fall season—a completely new sport for me. Not only did I learn the game, but I started following it on TV—something my younger self would never have believed! We made it to the playoffs, narrowly missing the semifinals, but the friendships and memories made it all worth it.

With my Soccer Team

One standout moment was a charity soccer match organized by a friend to raise funds for kids in Nigeria to attend school. It was 90 minutes of fierce competition, vibrant Afrobeats, and playful trash-talking—an unforgettable blend of fun and purpose. It inspired me, and I hope to participate in more activities like this in 2025 that combine enjoyment with a larger impact.

I also discovered pickleball, a fantastic (and forgiving) alternative to tennis, which still challenges me. Dancing was another unexpected highlight. I took classes in Bachata, Afrobeats and Amapiano, picking up a few moves to show off at parties and dance socials. Dancing brought joy, rhythm, and a sense of vibrancy to my year. At the gym, I focused on powerlifting with the StrongLifts 5x5 program, emphasizing core lifts like squats, deadlifts, bench presses, barbell rows, and overhead presses. This classic regimen, utilized by legends like Reg Park and Arnold Schwarzenegger, was a cornerstone of my training, with a clear focus on developing strength.

Toward the end of the year, I shifted to a more hypertrophy-focused program—Marcus Filly’s Functional Bodybuilding—which combines strength training with conditioning to reduce burnout and minimize the risk of injury. I got this recommendation from a friend, and it’s been a refreshing change thus far. I’ll continue with it in 2025 and potentially explore StrongLifts’ Hypertrophy focused workout program, the Stronglifts 5×5 Plus: Upper-body Hypertrophy Program.

Finally, there was skydiving. To celebrate my birthday, I took my first jump—a thrilling mix of exhilaration and sheer terror. Every second was unforgettable, and I loved it more than I could have imagined. I’m already planning at least two jumps in 2025 (it’s bloody expensive, though—holla at me if you know where to get discounts) and hope to convince a few friends to join me (though they’re not exactly eager—yet). Who knows? Maybe one day, I’ll even work toward earning a skydiving license.

Skydiving

Amid all this activity, my trusty watch tells me I logged 4,396,500 steps—an average of over 12,000 steps a day. But beyond the stats, 2024 reminded me of the profound connection between physical movement and mental clarity. Exercise wasn’t just a way to stay fit; it became my lifeline, a creative fuel, and the spark that kept me alive and more engaged with the world.

Professional Life: Putting Out Fires, Building Dreams, and Learning Through the Grind

The start of 2024 was a whirlwind of engineering work as I helped tackle fires at my job. Our company had been caught in the crosshairs of a large-scale nation-state hack, leaving critical systems vulnerable. The team worked tirelessly to fortify our systems, ensuring resilience against future attacks. The work was both challenging and rewarding, though it often came at a personal cost. There were nights when I stayed in the office until ungodly hours, running on fumes and pushing through burnout. My roommates frequently teased me about my relentless work ethic, and they weren’t wrong—it wasn’t great for my well-being. Still, I genuinely enjoyed the intensity and the tangible impact of the work. By the end of the fiscal year, my efforts were recognized with high commendations from my manager and a well-earned promotion. In 2025, I’m determined to continue working hard but with a healthier balance, prioritizing sleep, consistent meals, and sustainable habits, and focus to avoid the burnout I experienced this year.

Since stepping into my new role, I’ve faced a different kind of challenge: maintaining the same level of impact. As I feared, it sometimes feels like I’ve set an impossibly high bar with my earlier efforts, leaving me struggling to keep up. But growth often stems from discomfort, and I’m embracing the learning curve, determined to evolve and improve each day.

Beyond my 9-to-5, I launched this blog—a space to explore and share my thoughts, ideas, and learnings. This year, I wrote on various topics, including technology, sports, and politics. Writing has become my go-to method for deepening my understanding; explaining concepts to others not only helps me retain what I learn but also clarifies my thinking. In 2025, I’m eager to expand my horizons, learn even more, and turn those lessons into an even greater number of blog posts than in 2024. I hope you’ll be part of this journey, offering your insights and ideas along the way.

Subscribe now

One of my personal highlights this year was finally passing my professional engineering exams after an earlier stumble. This milestone, combined with reading more books than ever before and exploring diverse, often uncomfortable ideas, made 2024 a deep dive into knowledge. The more I learn, the more I realize how much I don’t know—a humbling yet invigorating revelation. My goal for 2025 is simple: to guzzle up as much knowledge and learning as possible.

Much of my year revolved around Lena, the startup I’ve been building with an incredible team. Lena’s mission is ambitious yet deeply needed: to transform education through low-tech learning and school management solutions, starting with schools in Nigeria. While we aimed to launch in 2024, we pushed the timeline to 2025. We weren’t satisfied that we had built something truly impactful for the schools we want to serve.

Lena Team Meeting

This setback wasn’t easy—especially with our runway shrinking fast. But perseverance paid off. Representing the team, I pitched Lena at Founders Live and secured $5,000 in funding. Thanks to our disciplined financial management, this funding buys us several months to refine our work. I’m incredibly proud of how the team has grown this year, with every member stepping up and delivering their best.

I can’t wait to see Lena make its mark, helping students in Nigeria and beyond unlock their potential. Our dream is to start with Nigeria, expand across Africa, and eventually serve communities worldwide that need accessible, low-tech educational tools. You can learn more about Lena and our vision on our website. 2024 was a year of hard lessons and big dreams for Lena, and 2025 promises to be the year we truly take flight.

Testing Lena's Mobile Applications

Relationships and Community: Strengthening Bonds, Building Bridges, and Learning from Mistakes

After graduating college, one of my biggest fears was that it would be incredibly challenging to make and maintain friendships in the so-called “real world.” Thankfully, that hasn’t been entirely true. While I miss the ease of meeting and connecting with people in college, I’ve come to appreciate the deliberateness and intentionality relationships require outside that bubble.

A highlight of this year was celebrating the graduations of some of my closest friends. These are the same friends with whom I studied late into the night during the chaos of the COVID-19 pandemic, often from 9 PM to 3 or 4 AM, over Zoom. We didn’t attend the same schools and were scattered across the country, but we stayed connected, supporting each other through victories, heartbreaks, and challenges. Seeing their growth and celebrating their achievements in Austin and Nashville was deeply fulfilling. Though we don’t talk as much now, our bond remains strong, and we prioritise being present for each other’s milestones—a habit we hadn’t cultivated while in college.

Locally, I’ve found joy in helping to build a small but tight-knit community in Seattle, where we go out for fun activities and hang out most weekends. Time outside work also allowed me to connect with people through soccer, pickleball, dance socials, and other activities. I’ve grown closer to my roommates, who generously let me host game nights at our house. This year, I hosted two big game nights (one drew over 80 people!), my soccer team for snacks and games, and a pirate-themed Christmas party complete with a treasure hunt for Santa’s lost “Parcel of Eternal Happiness.” To my surprise, the party (and hunt) was a massive success. I was particularly proud of our giving tree, which collected supplies, toys, and gift cards for a local Seattle charity supporting women and children experiencing homelessness. Special shoutout to my friend, Fortune, for being Santa, the Captain of the North Pole Fleet, during the party. In 2025, I hope to host even more events that bring people together for fun and meaningful causes.

Group Picture from Issa Game Night: Round 2

That said, I did drop the ball on one community-building effort this year. I had formed a partnership with a local club to host events but failed to follow through with proper plans. They were enthusiastic and welcoming, and I didn’t treat that relationship with the respect it deserved. I plan to reconnect in 2025 and make amends by delivering on those opportunities.

Treasure Hunters Teasing Out Clues from Santa at Christmas Under the Blag Flag

The highlight of my year, however, was when my parents visited Seattle. Spending time with them was a gift. As they approach new milestones, including my mom’s 60th birthday, I’m inspired by their unwavering energy, ambition, and commitment to learning. They’re still working toward new goals, transitioning into the next phase of their lives, and leading by example. Our relationship has grown significantly over the years. I’ve moved past the anger I once held, stemming from childhood traumas I attributed to their parenting. With time, I’ve come to see them as flawed, human, and deeply loving—parents who did their best with the knowledge they had. Today, I respect them more than ever and cherish the sacrifices they made for me and my sisters.

Dinner with my Friends, Cousin and Parents

Still, I have room for growth. This year, I let certain emotional triggers get the best of me, at times unfairly channeling that intensity toward those closest to me. It’s something I’m not proud of, and I deeply regret the hurt it caused. I’m committed to addressing these patterns and am actively using every tool at my disposal to improve. In 2025, I aspire to be a better roommate, friend, partner, son, brother, and teammate—to bring more empathy, patience, and kindness into every relationship in my life. Steve Jobs once said, “Some years you win, some years you build character.” For me, 2024 was undoubtedly a year of building character that challenged me to grow, reflect, and strive toward being a better version of myself. And in 2025, the work continues.

My Favorite Things from 2024

As I mentioned earlier, 2024 was a year of exploration—a time to immerse myself in art, music, literature, and more from various sources. I’d love to share some of the works that deeply resonated with me this year. You’ll likely notice a strong bias toward World War II history, a subject that has always fascinated me with its rich stories, complexities, and lessons. It’s a personal passion that fuels much of my curiosity. This year, I also found myself drawn to country music, discovering its ability to tell heartfelt and compelling stories. These selections aren’t necessarily new releases from 2024, but they’re the ones that made a lasting impact on me. I hope you enjoy this list and perhaps find a few recommendations that spark your own interest.

• Favorite Songs (10): Spotify Playlist

  • Tennessee Whiskey - Chris Stapleton

  • Eye Clear - Victor AD, Bella Shmurda

  • Wonderwall - Oasis

  • Tshwala Bam - TitoM, Yuppe, S.N.E, EeQue

  • Tell Somebody - Young Fathers

  • Talladega - Eric Church

  • January 9th - Black Sherif

  • Where the Wild Things Are - Luke Combs

  • Unonzani? - SunniChi

  • Learning to Fly - Tom Petty and The Heartbreakers

• Favorite Albums (10)

  • Utopia - Travis Scott (2023)

  • Random Access Memories (10th Anniversary Edition) - Daft Punk (2023)

  • Cowboy Carter - Beyoncé (2024)

  • A Head Full of Dreams - Coldplay (2015)

  • Ultra 85 - Logic (2024)

  • F-1 Trillion - Post Malone (2024)

  • The Great American Bar Scene - Zach Bryan (2024)

  • adedamola - Fireboy (2024)

  • The Stranger (Legacy Edition) - Billy Joel (1977)

  • MICHEAL (Deluxe) - Killer Mike (2023)

• Favorite Movies (5)

  • The Fall Guy (2024): A comedy starring Ryan Gosling and Emily Blunt, offering a behind-the-scenes look at the stunt performers who make blockbuster movies possible. The theme song, Unknown Stuntman by Blake Shelton is a gem.

  • Hit Man (2023): Glen Powell and Adria Arjona shine in this film based on the true story of a college professor working with the Houston police in the 1990s as a fake hitman to help prosecute murder solicitation cases.

  • On The Basis of Sex (2018): A moving portrayal of Ruth Bader Ginsburg’s groundbreaking fight against sex-based discrimination in the 1970s.

  • The Ministry of Ungentlemanly Warfare (2024): A hilarious Guy Ritchie film that tells the story of elite soldiers executing a covert strike against German forces in Africa during World War II.

  • The Wild Robot (2024): A heartwarming animated adaptation of Peter Brown’s novel, following Roz, a robot stranded on a remote island, as she learns to survive and connect with the world around her.

Honorable movie mention goes to The Six Triple Eight (2024). Released toward the end of the year, this compelling and inspiring film tells the story of the Black women in the Women’s Army Corps who played a crucial role in ensuring timely mail delivery during World War II. Their perseverance and dedication in the face of adversity make this a powerful and unforgettable tribute to their contributions.

• Favorite Books (10)

  • Poor Charlie’s Almanack: The Essential Wit and Wisdom of Charles T. Munger - Peter D. Kaufman

  • The Field Guide to Understanding ‘Human Error’ - Sidney Dekker

  • The Value of Others: Understanding the Economic Model of Relationships to Get (and Keep) More of What You Want in the Sexual Marketplace - Orion Taraban (This controversial and politically incorrect book offers bold insights into relationships and human behavior. While divisive, it made the list for the intriguing perspectives it brings to the table).

  • Flyboys: A True Story of Courage - James Bradley

  • Slow Productivity: The Lost Art of Accomplishment Without Burnout - Cal Newport

  • The Art of Racing in the Rain: A Novel - Garth Stein

  • Why Nations Fails - Daron Acemoğlu, James A. Robinson

  • The Road Less Traveled and Beyond: Spiritual Growth in an Age of Anxiety - M. Scott Peck

  • Never Split the Difference: Negotiating As If Your Life Depended On It - Chris Voss with Tahl Raz

  • How to Lie with Statistics - Darrell Huff, Irving Geis

Honorable book mentions go to How to Build a Car: The Autobiography of the World’s Greatest Formula 1 Designer by Adrian Newey, Surely You’re Joking, Mr. Feynman! (Adventures of a Curious Character) by Richard Feynman, Deep Work: Rules for Focused Success in a Distracted World by Cal Newport, and Stolen Focus: Why You Can’t Pay Attention—and How to Think Deeply Again by Johann Hari. While these weren’t first-time reads for me in 2024, I found myself returning to them throughout the year for their compelling insights.

A more controversial mention is Henry Ford’s My Life and Work. The book offers fascinating insights into Ford’s business philosophy and the strategies that built one of history’s most iconic automobile companies. However, its ideas are undeniably dated in many areas, and it takes a problematic turn with overt antisemitism. This led me to explore Ford’s troubling history regarding antisemitism, segregation, and his opposition to unions. You can read more here. While the book contains valuable lessons, I recommend approaching it critically, applying only its timeless principles while acknowledging its serious flaws.

Finally, my standout non-book mention goes to my Amazon Kindle Paperwhite Signature Edition (32 GB). Gifted to me for my birthday, it’s easily the best present I’ve ever received. Lightweight with a battery that seems to last forever, it has been my go-to for reading and listening to audiobooks without distractions. I can’t recommend it enough.

• Favorite TV Shows (5)

  • Masters of the Air (2024)

  • Band of Brothers (2001) - I’ll consider this the best TV show I have ever seen.

  • Peaky Blinders (2013 - 2022) - Still watching

  • Prison Break (2005 - 2017) - Still watching

  • The Pacific (2010)

I’ll be the first to admit that I’m not a big TV or movie watcher. These shows stood out to me as compelling stories worth recommending, even though I haven’t finished a couple of them yet. This list is a great place to start if you’re into engaging narratives and rich storytelling.

• Favorite Documentaries (5)

  • Five Came Back (2017)

  • Hitler and the Nazis: Evil on Trial (2024)

  • The Redeem Team (2022)

  • Get Gotti (2023)

  • Journey of an African Colony: The Making of Nigeria (2019)

• Favorite Podcasts (5)

  • The Joe Rogan Experience - Joe Rogan

  • Acquired - Ben Gilbert and David Rosenthal

  • Deep Questions with Cal Newport - Cal Newport

  • The Ezra Klein Show - Ezra Klein

  • Patrick Boyle: On Finance - Patrick Boyle

• Favorite Instrumentals (5): Spotify Playlist

  • This Is It - Blake Neely (Soundtrack, Masters of the Air)

  • Soar (Main Title Theme from ‘Masters of the Air’) - Blake Neely (Soundtrack, Masters of the Air)

  • What I Know - Adrian Walter

  • Main Titles (You’ve Been Called Back to Top Gun) - Harold Faltermeyer, Lady Gaga, Hans Zimmer, Lorne Balfe (Soundtrack, Top Gun: Maverick)

  • Main Titles from the HBO Miniseries Band of Brothers - London Metropolitan Orchestra, Michael Kamen (Soundtrack, Band of Brothers)

• Favorite Apps (5)

  • TickTick - Tasks, todo list, and schedule management

  • Monarch Money - Budgeting, planning, and financial management

  • Cronometer - Nutrition tracking

  • StrongLifts - Weighlifting and tracking for the Stronglifts 5 by 5 program

  • Readup - Read interesting articles

What am I Looking Forward to in 2025

As 2025 approaches, I’ve been mulling over several goals. I’ve already mentioned some; I’m still fine-tuning others to ensure they’re realistic and meaningful. I aim to dedicate more time to deep, focused work, learning, and personal projects. I desire to read at least 24 books (two per month), become a more effective communicator and storyteller, build consistent habits aligned with my aspirations, and maintain a sense of optimism—living with a positive mindset, laughter and a smile, even when challenges arise. Physically, I plan to focus on hypertrophy training and light running early in the year, shifting toward heavy running and training for a half-marathon personal record in the fall.

But enough about me—2025 promises to bring plenty of exciting things. Here’s what I’m most looking forward to:

• Books

  • Source Code by Bill Gates: A personal account of Gates’ childhood, early passions, and the experiences that shaped him into a transformative leader.

  • Hope: The Autobiography by Pope Francis: Tracing the Pope’s journey from his roots in Italy to his profound impact on the world today.

  • Changing My Mind by Julian Barnes: Exploring what it takes to change our minds, challenging assumptions and exploring the science and art behind such transformations.

  • Dream Count: A Novel by Chimamanda Ngozi Adichie: A compelling story about four women navigating love, longing, and pain. Adichie’s storytelling mastery makes this a highly anticipated read.

  • An Untitled Memoir by the “All Night Long” singer, Lionel Richie: The legendary singer recounts his life, from growing up in Alabama to becoming a global music icon.

  • This is For Everyone by Tim Berners-Lee. In my recent blog post about URL Encoding, I wrote briefly about Berners-Lee and his work creating the World Wide Web, HTTP, HTML, and URLs, which form the foundation of our current digital and communication landscape. In this book, the inventor of the World Wide Web tells the story of his groundbreaking invention and its global impact.

  • 1929: Inside the Crash by Andrew Ross Sorkin: A meticulously researched account of Wall Street’s most severe crash and the ensuing Great Depression.

  • The Next Day: Transitions, Change, and Moving Forward by Melinda French Gates: A rare glimpse into pivotal moments in Gates’ life and the lessons she’s learned.

  • Matriarch: A Memoir by Tina Knowles: Beyoncé and Solange’s mother shares her journey, highlighting the power of Black motherhood and resilience.

• Movies

  • Mission: Impossible - The Final Reckoning

  • Avatar: Fire and Ash

  • F1

  • Michael

  • Jurassic World Rebirth

  • Becoming Led Zeppelin

  • Paddington in Peru

• Music

  • 5ive by Davido (March 14, 2025)

  • No Sign of Weakness by Burna Boy

  • Possible new music from Cardi B and Rihanna

  • A hopeful release of J. Cole’s long-anticipated The Fall Off

• Video Games

I’m incredibly excited for Split Fiction by Hazelight Studio (March 6, 2025), a co-op game following two writers trapped in their own stories. As a fan of story-driven adventure games, I also hope for more news about Naughty Dog’s Intergalactic.

• Other Goals

In addition to the exciting releases and plans already mentioned, I have a few broader goals for 2025. I aim to deepen my cybersecurity knowledge and cultivate technical skills and expertise that align with my interests and aspirations in business, leadership, and entrepreneurship. On a personal level, I want to focus on building more profound, meaningful relationships and continuing to improve my emotional intelligence and resilience.

When it comes to reading, I’m looking forward to delving into timeless works like Mere Christianity by C. S. Lewis—purely for the joy of it—and revisiting classics like A Farewell to Arms by Ernest Hemingway. I also plan to explore books like Hit Refresh by Satya Nadella, among others that I’ll undoubtedly discover as the year unfolds. Depending on what captivates me, I’ll share more about these in my reflections at the end of 2025.

Another area of interest is the blog Hardcore Software: Learning by Shipping by Steven Sinofsky. Sinofsky’s insights into the challenges and triumphs of leading massive software projects—like Microsoft Office, Windows 7, and Windows 8—are invaluable for anyone striving to set ambitious goals and execute them effectively. I hope to glean valuable lessons for shipping products on time and achieving ambitious outcomes.

As the year progresses, I look forward to discovering new books, projects, and experiences that inspire growth, spark creativity, and enrich my understanding of the world.

Closing

2024 was a year of adventure, growth, and unforgettable memories. I’m grateful for the incredible people—friends, family, colleagues, and teammates—who made it all possible. Thank you, the reader, for taking the time to walk through my reflections. Here’s to a fantastic 2025 filled with joy, success, and Oscar-worthy moments. Godspeed.

Image Source: XKCD

Introduction

Have you ever clicked on a link or typed a web address only to see something strange in the URL? Maybe it looked like this:

https://internet.com/explore/search?query=what+is+a+good+application+of+N%C3%A4ive+Bayes+in+deep+learning

when your search query was: what is a good application of Näive Bayes in deep learning. Or perhaps you encountered something like this:

https://internet.com/travel/hotels/courtyard%20by%20marriot%20D%C3%BCsseldorf

when you were looking for Marriot hotels in Düsseldorf, Germany.

What are all those % signs and numbers? Are they some kind of secret code? In this blog post, we’re going to unravel the mystery behind these symbols and take a deep dive into URL encoding, ASCII, and the fascinating story of URLs themselves.

What are URLs?

A URL (Uniform Resource Locator) is the cornerstone of how we navigate the web. Simply put, a URL is an address that tells your browser where to find a specific resource, such as a webpage, an image, or a file. You can think of it as a digital version of a friend’s home address—it ensures that when you need something from your friend, you know precisely where to go to receive the article reliably.

A typical URL would look something like this:

https://www.internet.com:8080/path/to/resource?query=parameter#fragment

Here’s the breakdown:

Image Source: Wikipedia

• Protocol/URI Scheme(https://): Specifies how your browser should communicate with the internet server. Common protocols include http/https (secure http), ftp (file transfer), mailto (email address) and more. While network protocols are not the focus of this blog post, you can learn more about them in these articles: Types of Network Protocols and Their Uses, What is a network protocol?, Uniform Resource Identifier (URI) Schemes

• Domain(www.internet.com): Identifies the server hosting the internet resource. This is typically a human-readable name that maps to an IP address via DNS (Domain Name System). For a given domain, you can easily look up the corresponding IP address in DNS. For instance, on my local machine, I can use the host command or the nslookup utility to take a look at the IP addresses for Google servers. Public tools like DNS Checker and MX Toolbox are also handy for peeking at DNS records for domains.

• Port(:8080): Optional and specifies which port the server should use. The default for http is 80, and for https it’s 443. Other port numbers and the corresponding protocols are 21 for ftp, and 22 for ssh, etc. The port essentially specifies which application or service on a server to connect to via the URL. If the URL is your friend’s home address, the port number can be thought of as your friend’s room.

• Path(/path/to/resource): Indicates the specific location of the resource on the server. It’s like the folders and files on your computer.

• Query(?query=parameter): A set of key-value pairs used to pass information to the server. For example, a search query when we want to learn about Näive Bayes or a user’s preferences when making a request from the server.

• Fragment(#fragment): Refers to a specific section within the resource. For instance, it could be specific lines in a text file, a particular section or a bookmark on a webpage.

Where Did URLs Come From?

Image Source: CERN

In 1992, Tim Berners-Lee introduced the concept of URLs alongside the HTTP protocol and HTML as a way for researchers to share and access documents easily.

Berners-Lee has already proposed the ideas of the World Wide Web. However, for this network to function, it needed a standardized way to identify and locate resources. To address this need, he proposed the idea of the URL to serve as a “document identifier.” The URL became one of the three core components of the web, alongside HTML (for structuring documents) and HTTP (for transferring them).

Initially, URLs were simple and primarily used to point to static files hosted on servers. Over time, they evolved alongside the web into a versatile system supporting dynamic content, user input, and even encrypted communication. Throughout this evolution, the central principle of universality—the idea that URLs should work on any device and in any context—has stood the test of time.

ASCII and Its Role in URL Encoding

To understand the “strange” encodings we see in URLs, we need to look back at the history of ASCII. The American Standard Code for Information Interchange (ASCII) was developed in the 1960s as a standardized way for computers to represent text characters. Before ASCII, there were numerous incompatible encoding systems, which made it challenging for different systems to communicate. ASCII changed that by providing a universal 7-bit character set, which could represent 128 unique characters.

This 128-character set included:

• Printable characters: Uppercase (A-Z), lowercase (a-z), digits (0-9), and symbols like @, #, and $, etc.

• Control characters: Instructions for managing text streams, such as newline (\n) and tab (\t).

ASCII’s simplicity and universality made it the foundation for early computer systems and networks, including the internet. However, its biggest limitation was its inability to represent non-English characters, like é, ß, or 中, as well as other writing systems like Cyrillic, Arabic, and Chinese. The biggest reason for this limitation is because, at the time it was invented, memory and processing power were incredibly expensive; hence, every bit mattered. By sticking to 7-bits and, thus, 128 characters, ASCII struck a balance between functionality and efficiency. It was small enough to fit into the limited storage and memory of the time yet comprehensive enough to provide a range of characters to work with. Likewise, it was a light-weight, simple, easy-to-implement solution and universal (at least for English-speaking developers).

Expanded Character Sets (Beyond ASCII)

Image Source: XKCD

As the internet connected the world, the need for a broader character set became apparent. This led to the development of Unicode, which could represent virtually every character in every language. Unicode works with multiple encodings, such as UTF-8, UTF-16, and UTF-32 (You can read more about them here: Difference between UTF-8, UTF-16 and UTF-32 Character Encoding? Example)

UTF-8 is the most widely used encoding on the web today. It is backwards-compatible with ASCII, meaning that all ASCII characters retain their original binary values, while non-ASCII characters are represented using additional bytes. For example:

• The ASCII character A remains 01000001 in binary under UTF-8.

• The Unicode character é is represented as 11000011 10101001 in UTF-8.

Despite Unicode’s dominance in modern text representation, URLs remain constrained to ASCII due to backward compatibility and simplicity. Early internet protocols, including URLs, were built on ASCII. Changing this foundation would disrupt countless systems and applications.

To address this, the solution was percent-encoding, or URL encoding, which allows characters outside the ASCII range to be safely represented in URLs.

URL Encoding

URL encoding ensures that any character—whether it’s unsafe, reserved, or non-ASCII—can be safely transmitted in a URL. Here’s how it works:

1. Identity Characters to Encode:

   • Reserved Characters: Characters with special meanings in URLs (e.g., ? to start a query string, & to separate parameters, / to separate path components, etc.) must be encoded when used outside their context.

   • Unsafe Characters: Characters like spaces, <, >, {, }, etc., are unsafe because gateways and transport agents might modify them. Encoding prevents such misinterpretation.

   • Non-ASCII Characters: These characters, which fall outside the ASCII set, must be encoded for compatibility across systems.

2. Convert Characters to Hexadecimal ASCII:

   • Each character is replaced with a % followed by its two-digit hexadecimal value. (Use this handy URL Encoding Reference) mapping characters to their hexadecimal values.

Details Examples of URL Encoding

Reserved/Unsafe Characters in Query Strings

Original: https://internet.com/search?q=C++&C Programming
Encoded:  https://internet.com/search?q=C%2B%2B%26C%20Programming

Here,

• + becomes %2B (reserved character).

• & becomes %26 (reserved character).

• Space ( ) becomes %20 (unsafe character).

Non-ASCII Characters

Original: https://internet.com/profile?name=Günter Hernández François
Encoded:  https://internt.com/profile?name=G%C3%BCnter%20Hern%C3%A1ndez%20Fran%C3%A7ois

In this case,

• ü is encoded as %C3%BC.

• á is encoded as %C3%A1.

• ç is encoded as %C3%A7.

• Space is encoded as %20.

Below are examples of encodings for some reserved characters:

And correspondingly, for some non-ASCII Unicode characters:

Conclusion

URL encoding is a critical mechanism that allows URLs to safely transmit a wide range of data while maintaining compatibility with the internet’s ASCII-based foundation. So, the next time you see %20 or %C3%A9 in a URL, don’t panic—it’s just the internet’s way of speaking a language that devices all around the globe can understand.

References

While I have cited a handful of sources in this blog post, below are some other resources that I used to write it and that can be useful for your further learning.

• HTML URL Encoding Reference

• HTML ASCII Reference

• Punycode: A Bootstring encoding of Unicode for Internationalized Domain Names in Applications (IDNA)

• What Happens When You Enter a URL In Your Browser of Choice

• The birth of the Web

• Uniform Resource Identifier (URI): Generic Syntax

• URL Encoding and what characters are valid in a URI

• The Blessing of Unicode

• Reserved and excluded characters

• Types of Network Protocols and Their Uses

• What is a network protocol?

• What is a URL (Uniform Resource Locator)?

• The components of a URL

• Uniform Resource Identifier (URI) Schemes

• How domain name is transferred in IP address?

• Difference Between Domain Name and URL

• What is the difference between a website, domain name and a URL?

• port number

• What Are Ports?

• Difference Between IP Address and Port Number

• Query Parameters and URI Fragments in URLs

• The History of the URL

• Document identifiers

• Computer Mail Meeting Notes

• How The URL Was Built

• Information Management: A Proposal

• Timeline of ASCII History

• Unicode: The Journey From Standardizing Texts to Emojis

Image Source: Recursion: The Pros and Cons
Original creator unknown

Introduction

When I was in elementary school, I lived in a dormitory with my classmates, and we all had daily chores. These chores ranged from sweeping to mopping and other tasks to keep our hostels and school tidy. Back then, I wasn’t exactly a fan of cleaning. I saw beauty in disorder and only did my duties to avoid trouble. However, one item always captivated me: the RoyalMop bucket.

At first glance, it was just an ordinary blue mop bucket, like any other. But what fascinated me was the label on the side. The label showed a picture of the mop bucket—including the label itself. This meant the image on the label contained another image of the mop bucket, which in turn had another label, and so on. In theory, this created an infinite loop of mop bucket images, spiraling deeper and deeper into itself.

I was mesmerized by this design. The bucket’s label was like a window into an endless series of smaller and smaller buckets, each one a reflection of the larger one. It was a simple object, yet it held an infinite pattern within it—so each time I saw it, I couldn’t help but pause and stare.

Years later, I found myself thinking about that mop bucket when learning about recursion in computer science. Like the label on that bucket, recursion involves a process that repeats within itself, potentially ad infinitum. It’s the same concept: something self-referential, infinitely repeatable. While searching for resources while writing this blog post, I stumbled upon images of the same RoyalMop bucket that had fascinated me years ago. The quality of the image may not be high-definition, but the recursive design still shines through.

Take a look for yourself. Notice how the label depicts the bucket, and within that, another label, and so on. It’s a real-world example of what happens under the hood of recursive implementations in computer science.

Image Source: Celplas Mopping Bucket

Recursion in Computer Science

Now, let’s jump from my elementary school days to the world of integrated circuits, machine code, and programming. In this context, recursion is a programming technique where a function calls itself to solve smaller instances of the same problem.

A recursive approach works by breaking a larger problem into smaller, more manageable pieces, each of which is a simpler version of the original task. For example, imagine you have a function that can print a single character, and your task is to print the phrase hello world. You can break this problem into smaller chunks: h, e, l, and so on. The function can then handle each character one by one until the entire phrase is printed.

This way, recursion simplifies complex tasks by tackling them piece by piece—just as that infinite series of mop bucket images breaks down into smaller and smaller reflections of itself.

In Python, this concept could be implemented using a function print_one_character to print a single character:

def print_one_character(char):
    # This function prints a single character
    print(char, end="")

We can then use this function in a recursive function, print_string_recursively, which continues calling print_one_character until all the characters in the phrase are printed:

def print_string_recursively(string):
    # If the string is empty, stop the recursion (we are done printing)
    if len(string) == 0:
        return
    else:
        # Call `print_one_character` to print the first character of the string
        print_one_character(string[0])
        
        # Recursively call `print_string_recursively` on the remaining part of the string
        # For example, after printing `h`, the function is called on 'ello world' for `e` to be printed next and so on
        print_string_recursively(string[1:])

Using this simple premise of breaking a problem down into smaller, similar subproblems, recursion becomes an invaluable tool, especially in problems like searching through file directories, sorting data with algorithms like merge sort, or working with hierarchical structures like trees. Its ability to handle self-similar, nested tasks with elegance and simplicity makes it indispensable in many fields of computer science, from data processing to AI algorithms.

Key Components of Recursion

A recursive implementation relies on two critical components: the base case and the corresponding recursive case. The base case is crucial because, while recursion allows a function to repeat indefinitely, we typically want it to stop once the problem is solved. The base case defines the condition under which the recursive function stops calling itself, preventing infinite recursion. Without a base case, the function would continue to call itself indefinitely, eventually causing a stack overflow (we’ll dive into that later 😉).

Here’s an example using a factorial function:

def factorial(n):
    if n == 0:  # Base case
        return 1
    else:
        return n * factorial(n - 1)  # Recursive case

In this example, the base case occurs when n == 0. At this point, recursion stops, and the function returns 1. Every recursive function needs at least one base case to ensure the recursion eventually halts.

On the other hand, the recursive case is where the function calls itself with a modified input—typically a smaller or simpler version of the original problem. In the factorial example, the recursive case is n * factorial(n - 1). Each time the function calls itself, it reduces the problem by decrementing n by 1, gradually moving toward the base case. This recursive structure breaks the problem into smaller parts, eventually converging on the base case.

The interaction between the base case and the recursive case ensures that recursion solves progressively smaller subproblems and then combines the results when the base case is reached. For example, if we calculate factorial(4), the execution would look like this:

factorial(4): return 4 * factorial(3)
    factorial(3): return 3 * factorial(2)
        factorial(2): return 2 * factorial(1)
            factorial(1): return 1 * factorial(0)

Here, our base case, factorial(0) ", yields 1. Once we hit this final result, we combine it with the previous results. factorial(1) computes 1 * 1 = 1, which we then use to compute factorial(2) as 2 * 1 = 2. Next, we compute factorial(3) as 3 * 2 = 6, and finally, we compute factorial(4) as 4 * 6 = 24`, giving us the final combined result.

Another Example: Fibonacci Sequence

Image Sources: Fibonacci Numbers, Exploring the Fascinating World of the Fibonacci Sequence

Here’s another example of a recursive implementation: the Fibonacci sequence, which is a common use case of recursion in programming.

The Fibonacci sequence is a series of numbers where each number is the sum of the two preceding ones. The sequence typically starts as: 0, 1, 1, 2, 3, 5, 8, 13...

The Fibonacci sequence can be defined recursively, because each Fibonacci number is the sum of the two preceding numbers, except the first two numbers in the sequence: fibonacci(0) = 0 and fibonacci(1) = 1. These two serve as our base cases. Here’s the function definition below:

# recursive fibonacci function
def fibonacci(n):
    if n == 0:  # Base case 1
        return 0
    elif n == 1:  # Base case 2
        return 1
    else:
        return fibonacci(n - 1) + fibonacci(n - 2)  # Recursive case

Now, let’s evaluate the cases:

1. Base Cases:

   • The base cases stop the recursion from going on indefinitely. We know that the first two numbers in the Fibonacci sequence are predefined as fibonacci(0) = 0 and fibonacci(1) = 1.

   • When n reaches either 0 or 1, the function will return 0 or 1 respectively, stopping further recursion for that branch.

   • We could also simplify the base case by returning n when n == 0 or n == 1 or n < 2. However, in this example, we explicitly define two base cases for clarity.

2. Recursive Case

   • The recursive case is where the function calls itself. For all n > 1, the function calls itself twice: fibonacci(n - 1) and fibonacci(n - 2), summing these two calls to get the `n’th Fibonacci number.

   • This recursive structure reflects how the Fibonacci sequence is built, as each number in the sequence is the sum of the previous two numbers.

Let’s break down the execution:

What if we want to compute fibonacci(3):

1. Intiail call: fibonacci(3)

   • Since n is not 0 or 1, we enter the recursive case: fibonacci(3) = fibonacci(2) + fibonacci(1).

2. Evaluate fibonacci(2):

   • Similarly, fibonacci(2) triggers the recursive case: fibonacci(2) = fibonacci(1) + fibonacci(0).`

3. Base cases fibonacci(1) and fibonacci(0)`:

   • fibonacci(1) hits the base case and returns 1.

   • fibonacci(0) also hits the base case and returns 0.

4. Unwinding the recursion:

   • Now, we combine the results, starting from the base cases:

     • fibonacci(2) = 1 + 0 = 1.

     • And finally, fibonacci(3) = fibonacci(2) + fibonacci(1) = 1 + 1 = 2.

The final result of fibonacci(5) is 5.

Real-world Applications of Recursion

Recursion has various real-world applications across multiple domains, from sorting algorithms to tree data structure and dynamic programming. I’ll discuss some of these applications in detail below:

Divide and Conquer Algorithms

Divide and conquer algorithms are almost inherently recursive, as the logic is to “divide” a problem into smaller subproblems, which are easier to solve, and then “conquer” each subproblem recursively before combining the results to get the final solution. Two popular examples of divide and conquer algorithms are merge sort and quick sort:

Merge Sort: This algorithm divides an unsorted list into two halves, recursively sorts each half, and then merges the sorted halves. For example, if we need to sort a list of numbers, merge sort recursively splits the list in half until it reaches sublists of size 1, which are inherently sorted, and then merges them. Below is a good graphic on how this algorithm proceeds:

Image Source: Understanding Merge Sort Algorithm with Java Source Code

Quick Sort: This algorithm works by selecting a “pivot” element from a list of items, partitioning the remaining elements into two sublists based on whether they are less than or greater than the pivot, and then recursively applying the same process to the sublists. Once the base case (an empty or single-element list) is reached, the recursion unwinds, and the fully sorted list is returned. This recursive partitioning enables quicksort to handle large datasets efficiently.

Image Source: Quick Sort Algorithm

Tree Traversals

Recursion is also a powerful tool for traversing tree structures such as binary trees, binary search trees, and decision trees. In tree traversal, each node in a tree has a similar structure, consisting of a value and links to its child nodes. Recursion is ideal for this task, as it allows the traversal function to explore nodes in a specific order. The base case will be an instance where a node has no children, while the recursive case involves calling the traversal function on the children of the node—thus moving through the tree. It is pretty elegant how tree traversals are implemented recursively. You can read more about them in these articles: 4 Types of Tree Traversal Algorithms, Navigating Trees: In-Depth Look at Traversal Algorithms.

Dynamic Programming

In dynamic programming, recursion is often combined with memoization to solve problems more efficiently by avoiding redundant computation. In cases where recursion might be inefficient due to repeated calculation of the same subproblem, dynamic programming offers an optimized approach by storing the results of expensive function calls and reusing them when needed.

A classic example of recusion and memoization is the Fibonacci sequence. In a naïve recursive implementation of the Fibonacci sequence, the function repeatedly recalculates the same Fibonacci numbers, leading to exponential time complexity. For example, to calculate fibonacci(5), the recursive function calculates fibonacci(4) and fibonacci(3), but to compute fibonacci(4), it again calculates fibonacci(3), creating a lot of redundant work.

To optimize this, memoization stores the result of each Fibonacci number once it’s computed, so the function doesn’t have to recompute it in future calls. Here’s an optimized version using recursion and memoization:

def fibonacci(n, memo={}):
    if n in memo:
        return memo[n]
    if n == 0 or n == 1:
        return n
    else:
        memo[n] = fibonacci(n - 1, memo) + fibonacci(n - 2, memo)
        return memo[n]

By storing the results of subproblems in the dictionary (memo), this version of the Fibonacci function ensures that each Fibonacci number is only computed once, reducing the time complexity from exponential to linear.

Recursion and memoization are frequently used in a wide range of dynamic programming problems such as knapsack problems, minimum edit distance (used in spell checkers or DNA sequence alignment), and longest common subsequence (used in version control systems). In each case, recursion breaks the problem into overlapping subproblems, and memoization ensures that these subproblems are not recomputed, greatly enhancing efficiency.

Other Application

Beyond the abovementioned ones, recursion is used in numerous other real-world applications. Some examples include backtracking algorithms (used in Sudoku, Maze solvers, etc.), data structure operations such as linked lists or trie data structures, web crawlers (used in search engines), expression evaluations (such as parsing nested parentheses and multiple operations in mathematical expressions), pathfinding algorithms, network protocols, robotics and simulations, image processing, social networks, and many more.

Common Pitfalls in Recursion

Image Source: SMBC Comics: Recursion

While recursion is a powerful tool extensively used in programming, it comes with its own set of challenges and pitfalls. The most common issues that arise with recursive algorithms are stack overflow (not the popular Q&A website for developers—even though I mischievously linked it above 😈) and efficiency drawbacks.

Stack Overflow

Before diving into stack overflow, let’s understand how the call stack works. Think of the call stack as a stack of plates in a kitchen. When you wash dishes, each time you finish cleaning a plate, you place it on top of the stack. As you keep washing more plates, you continue stacking them on top. When you’re done, you take the top plate off first, then the next, and so on—removing them in reverse order. This is exactly how the call stack works in programming: each new function call gets “stacked” on top of the previous one, and when a function completes, it is “popped” off the stack in a last-in, first-out (LIFO) manner. You can read more about the call stack in this article.

Now, imagine writing this blog post. I started by creating an outline of the concepts I wanted to cover, breaking each central idea into sections I could flesh out. I tackled each section individually, starting with the Introduction. Once I finished a section, I set it aside (popped it off my mental “stack”) and moved on to the next section. Each time I finished one section, I could remove it from my mental stack and focus on the next one.

But what if I tried to write the entire blog all at once without breaking it down? Imagine trying to tackle every section and paragraph simultaneously without giving each part my entire focus. My mind would get cluttered with all these unfinished pieces. I’d eventually get overwhelmed because my “mental stack” would become too full. This is similar to what happens in programming when too many function calls are stacked on top of each other due to deep or infinite recursion.

In the case of recursion, a new “task” is added to the call stack each time a function calls itself, just like focusing on each section of the blog. Each recursive call represents a piece of the problem the function is trying to solve. As the program finishes working on a recursive call (just like completing a section), it removes it from the stack and moves on to the previous function that called it.

If there’s no clear base case in the recursive function, or if the input size is too large, the call stack keeps growing without a way to reduce it. This is like trying to write more and more sections of the blog without finishing any of them. Eventually, I run out of focus and capacity to keep track of what I’m working on. That’s a stack overflow. Essentially, too many function calls consume memory on the system’s call stack, and the stack exceeds its limit, causing an error.

For example, consider a recursive function without a base case:

def infinite_recursion(n):
    print(n)
    return infinite_recursion(n + 1)  # No base case to stop the recursion

In the above code, the function infinite_recursion keeps calling itself indefinitely, consuming memory on the stack until the program crashes with a stack overflow error. This is why defining a clear and reachable base case is critical in any recursive function.

Even with a proper base case, deep recursion can cause a stack overflow if the input size is too large. For instance, computing large values in the Fibonacci sequence using a naïve recursive implementation can result in deep recursive calls that exhaust the stack:

def fibonacci(n):
    if n == 0 or n == 1:  # Base cases
        return n
    return fibonacci(n - 1) + fibonacci(n - 2)

For large values of n, such as fibonacci(2000), the above function will make an enormous number of recursive calls, leading to stack overflow due to the depth of recursion.

Efficiency

Another concern with recursion is its efficiency. Recursive solutions can be less efficient than iterative solutions due to the overhead of repeated function calls. Each recursive call adds an entry to the call stack, which consumes memory and takes time to set up. This overhead can significantly impact performance, especially for problems with deep or highly repetitive recursive calls.

For example, a simple loop can often achieve the same result as a recursive function with less memory usage and faster execution time. Let’s compare two implementations of factorial:

Recursive Factorial

def factorial_recursive(n):
    if n == 0:
        return 1
    return n * factorial_recursive(n - 1)

Iterative Factorial

def factorial_iterative(n):
    result = 1
    for i in range(1, n + 1):
        result *= i
    return result

The iterative approach avoids the overhead of function calls and potential stack overflow, making it more memory-efficient. However, recursion offers a cleaner, more intuitive approach in some cases, especially when working with problems with a natural recursive structure (like tree traversals).

The choice between recursion and iteration often depends on the complexity of the problem and the size of the input. For example, while recursion is elegant and simple for problems like tree traversal or divide-and-conquer algorithms, iterative solutions might be more suitable for scenarios with high memory constraints or large datasets.

These are good articles to learn more about efficiency as related to recursions: Why Recursion Is Less Efficient Than a Loop — Programming Word of the Day, Performance of recursive functions (and when not to use them)

Tail Recursion Optimization

Image Source: Optimization of tail recursion in R

Tail recursion is a specific type of recursion in which the recursive call is the last operation in the function. This seemingly minor distinction is quite powerful because it enables some programming languages to optimize recursive calls to prevent stack overflow and reduce memory usage. This optimization is known as Tail Call Optimization (TCO).

What is Tail Recursion?

In a normal recursive function, each recursive call creates a new stack frame that holds information about the current function’s execution context (local variables, return address, etc.). As recursive calls accumulate, each call leaves behind an unresolved task in the stack, leading to an ever-growing stack. This is why, in deep recursion, you often encounter stack overflow errors—the call stack grows beyond its limit.

However, in a tail-recursive function, the recursive call is the very last operation that the function performs. When a recursive call is the last thing a function does, there’s no need to keep the current stack frame. This allows the language or compiler to reuse the same stack frame for the next call, essentially transforming the recursion into an iteration under the hood.

Example: Tail Recursion vs. Normal Recursion

Let’s compare a simple recursive function to its tail-recursive version. Consider a function that calculates the sum of the first n numbers.

def sum_normal(n):
    if n == 0:  # Base case
        return 0
    return n + sum_normal(n - 1)  # Not a tail call because of the addition after the call

In this example, after the recursive call sum_normal(n - 1) "is made, the function still needs to add n to the result. This creates a chain of unresolved additions that each need their own stack frame. For large values of n`, this chain can quickly lead to a stack overflow.

Now, let’s rewrite this function using tail recursion.

def sum_tail_recursive(n, accumulator=0):
    if n == 0:  # Base case
        return accumulator
    else:
        return sum_tail_recursive(n - 1, accumulator + n)  # Tail call

In this version, the recursive call sum_tail_recursive is the very last thing that happens in the function. All the necessary work (adding n to the accumulator) is done before the recursive call. There’s no further computation needed once the recursive call returns. This is the hallmark of a tail-recursive function.

How Tail Call Optimization Works

In a tail-recursive function, once the final recursive call is made, the current stack frame is no longer used. In languages that support Tail Call Optimization (TCO), the compiler can recognize that it doesn’t need to keep the current stack frame around. Instead, it can reuse the current stack frame for the next recursive call, effectively converting the recursion into an iterative process without needing additional memory.

Since TCO allows the compiler to reuse the current stack frame instead of creating a new one for each recursive call, the call stack size doesn’t grow with each recursive call. This prevents stack overflow, even if the function makes many recursive calls.

For example, in a language with TCO support (like Scheme, Haskell, Scala, and Clojure), a tail-recursive function like the one above will consume a constant amount of stack space, regardless of the depth of recursion. O, even if a function is written in a tail-recursive form, it won’t receive the same stack-saving optimizations as in other languages.

Visualizing Tail Recursion vs. Normal Recursion

Imagine you’re trying to count the number of steps on a staircase:

• Normal Recursion: With a non-tail-recursive approach, every time you take a step, you leave a “marker” behind so you can retrace your steps on the way back. For instance, if you walk up 100 steps, you’ll leave 100 markers. When you reach the top, you have to return to each step to pick up your markers in reverse order. This is similar to normal recursion—every recursive call leaves behind some unfinished work (an unresolved operation) that needs to be revisited later. This ongoing “memory” of unfinished tasks builds up in the call stack, leading to potential stack overflow.

• Tail Recursion: Now imagine a different approach where you don’t leave any markers behind. Instead, at each step, you update a running count and move to the next step without needing to remember where you’ve been. When you reach the top, you already have the final count—there’s no need to retrace your steps. This is how tail recursion works: each recursive call performs its work completely before moving on without leaving unresolved operations behind on the stack. This efficiency prevents the stack from growing, allowing you to handle large recursions without running out of memory.

Conclusion

I hope you enjoyed reading this blog post and learned something new. For the closing, let’s play a little game that might keep you staring at this page forever. Have fun 😅

def conclusion():
    print("Recursion is great for breaking down problems into smaller parts. But if you forget a base case, you might be stuck here forever…")
    conclusion()  # Oops, we did it!

References

While I have cited a handful of sources in this blog post, below are some other resources that I used to write it and that can be useful for your further learning.

• Project Python: Recursion

• Recursion: The Pros and Cons

• How to Think Recursively | Solving Recursion Problems in 4 Steps

• Fibonacci Numbers

• Merge Sort – Data Structure and Algorithms Tutorials

• Memoization

• What is Backtracking Algorithm? Types, Examples & its Application

• Tail Call Optimization: What it is? and Why it Matters?

• Tail-recursive algorithms

• Tail Call Optimisation in C

• What is Tail Recursion

• This is a Better Way to Understand Recursion

• 5 Simple Steps for Solving Any Recursive Problem

• Recursion

• Reading 10: Recursion

• Recursion in Python: An Introduction

• Recursion

• Recursion and Looping

• Recursion vs. Looping in Python

• Different Techniques for Iterating over Data

• Difference between Recursion and Iteration

• Benchmark: Recursion vs Iteration

• A cheap trick to speed up recursion in C++

• Optimizing Recursive Algorithms: Techniques to Reduce Space Complexity

Image Source: ScotWork USA

Politics is one of the most divisive topics in modern conversations. Many suggest avoiding political discussions altogether, especially in the workplace, as highlighted in articles like 4 Compelling Reasons To Never Talk Politics At Work and The perils and pitfalls of discussing politics with family and friends. Others argue that open dialogue is essential for democracy, emphasizing the importance of talking—even when we disagree. Research also shows that nearly half of Americans have stopped talking politics with someone, and both Democrats and Republicans report that talking about politics with people who disagree is stressful, underscoring how difficult these conversations can be.

So why am I writing about politics? For me, it all began with a friend who is deeply politically engaged. Our conversations were often dominated by his political opinions. I usually let him speak because I found these discussions to be learning opportunities. One day, however, he shared a strong take on an issue that I believed was far more nuanced, which led to a debate. I quickly realized that while I disagreed with him, I lacked sufficient knowledge to push back effectively. This prompted me to dive into political history—not just to challenge him but also to understand the roots of our disagreements.

Ironically, my quest to “prove him wrong” deepened my interest in politics, particularly in how political systems evolved into what they are today. I figured sharing some of this history would create an engaging blog post. And what better time to explore political history than during a U.S. Presidential election year?

Over time, my friend and I found that we agreed on more than we initially thought—our debates helped us reach that understanding. In our case, talking about politics brought us closer rather than dividing us further. But that’s besides the point. A larger question loomed in my mind: Why are political conversations so polarized? Why is there a divide between “left” and “right” in the first place? In this post, I’ll explore the origins and evolution of left and right-wing politics. My goal is to provide some historical context, allowing you, the reader, to understand better these ideologies and how they shape today’s political landscape.

It’s important to note that the ideologies and history discussed in this article are primarily relevant to Western countries. In other parts of the world, such as Nigeria, where I’m from, politics often takes on different shapes and forms that may not be represented here.

With that said, let’s delve into the history of left and right-wing politics.

It Started in France

Image Source: American Founding Observations of the French Revolution That Influenced the United States Constitution and Governing

Like champagne, the metric system, and cinema, the political terms “left-wing” and “right-wing” also originated in France—specifically during the French Revolution. In 1789, France, under the absolute rule of King Louis XVI, was on the verge of collapse. The country faced a massive national debt and a dire economic situation. Compounding the issue, the wealthiest members of society paid very little in taxes. As calls for political reform grew louder, the King convened the Estates General, a representative national body that had not met since 1614. The Estate-General consisted of 300 representatives from the First Estate (the clergy), 300 from the Second Estate (the nobility), and 600 from the Third Estate (the commoners).

When they gathered at Versailles, the nobility and clergy sat on the King’s right while the commoners sat on his left. The goal of the meeting was to draft a new constitution to address the mounting crisis.

Image Source: The left vs right battle: 1. the meaning of left and right

In the newly recognized National Assembly, the more revolutionary members who sought radical changes sat on the left side of the president. They advocated for the overthrow of the monarchy and the establishment of a republic. This group became known as the left wing. On the right side of the Assembly, supporters of the monarchy and the existing social order took their seats. They were more conservative, favoring moderate reforms and the preservation of religious and aristocratic traditions. This group became known as the right wing.

Thus, broadly speaking, the terms “left-wing” and “right-wing” came to represent two ideological groups: the right wing, which favors conservatism and the preservation of existing societal structures, and the left wing, which advocates for more radical changes and societal reform.

Going Around the World

After the French Revolution, the ideological divide between the left and right quickly spread beyond France’s borders. In the early 19th century, Europe was a hotbed of political and social change, and the concepts of “left-wing” and “right-wing” became essential tools for understanding the growing tensions. The Napoleonic Wars (1803–1815) accelerated this spread, as Napoleon’s expansionist campaigns brought revolutionary ideas, including concepts of citizenship and equal rights, to countries across Europe. As monarchies were restored after Napoleon’s defeat, the terms left and right solidified: the left symbolized the drive for republicanism, equality, and the dismantling of feudal systems, while the right represented the desire to return to the status quo, preserving monarchies, aristocratic privileges, and church power.

Mid-19th Century: The Rise of Social Movements

By the mid-19th century, industrialization had transformed Europe, creating a growing divide between the wealthy elites and the burgeoning working class. Left-wing ideas gained further traction during this period, especially with Karl Marx’s Communist Manifesto in 1848. Marx’s call for a classless society and the abolition of private property became a rallying cry for many labor movements, aligning with the left’s push for workers’ rights, social equality, and government intervention to address economic disparities. The revolutions of 1848, also known as the “Springtime of Nations,” saw uprisings across Europe as people demanded constitutional governance, freedom of speech, and economic reforms. Though most of these revolutions were unsuccessful, they left an indelible mark on European politics, cementing the left as the voice of the working class and reformers.

On the other side, the right wing became more deeply associated with the defense of property rights, monarchies, and the interests of the upper classes. Conservatives viewed the rapid social changes brought about by industrialization with suspicion, fearing that upheavals could lead to instability. In response to the revolutionary wave, conservative leaders like Austria’s Prince Metternich and later, Otto von Bismarck in Germany, sought to maintain political order by suppressing socialist movements while enacting limited reforms to appease the masses. Bismarck’s creation of a welfare state, for example, was designed to undermine support for socialist parties by addressing workers’ needs without fundamentally altering the capitalist system.

Late 19th Century: Left and Right Take on New Dimensions

In the late 19th century, the left-right divide continued to evolve, particularly as Europe expanded its global influence through colonization. The left, still championing the rights of the working class, began to incorporate ideas about national liberation and anti-imperialism. Many left-wing groups supported independence movements in colonies, seeing the struggles of colonized peoples as an extension of their fight against oppression. Conversely, the right became closely associated with nationalism and imperialism, advocating for expanding European empires to bolster national pride and economic power.

The formation of political parties based on these ideologies became a defining feature of late 19th-century politics. In the United Kingdom, for example, the Labour Party emerged as a voice for workers, aligning with the left’s push for social and economic reform. Around the same period, the Conservative Party solidified its position as the defender of traditional institutions, private property, and national interests.

Early 20th Century: The Ideological Clash Intensifies

Image Source: The Russian Revolution at 100

The early 20th century saw the left-right divide take on an even greater significance, especially in the wake of World War I. The Russian Revolution of 1917, which led to the establishment of the world’s first communist state, was a pivotal moment for the left-wing movement globally. The revolution inspired leftist movements around the world to pursue similar aims—namely, the overthrow of capitalist systems in favor of socialist or communist alternatives. Throughout Europe, left-wing parties gained popularity, particularly in countries suffering from economic hardship and political instability following the war.

Meanwhile, the right wing in Europe became more polarized. Traditional conservative parties continued defending capitalism and social hierarchies, but far-right movements emerged, rejecting socialism and liberal democracy. The rise of fascism in Italy under Mussolini and Nazism in Germany under Hitler represented the extreme right. These movements sought to establish authoritarian regimes based on ultranationalism, militarism, and the rejection of socialist principles. Mussolini and Hitler capitalized on widespread disillusionment with post-war democracy and economic instability, offering a radical alternative to the left’s vision of classless, egalitarian societies.

The 1930s, therefore, became a period of intense ideological conflict, with left-wing communism and socialism clashing with right-wing fascism and conservatism. The Spanish Civil War (1936–1939) was a microcosm of this broader global struggle, with left-wing republicans and anarchists fighting against the right-wing Nationalists, who were backed by fascist Italy and Nazi Germany. Ultimately, the Nationalists won, establishing a dictatorship under Francisco Franco, but the Spanish Civil War foreshadowed the more significant conflicts.

World War II and the Cold War

World War II (1939–1945) marked the peak of the ideological struggle between left and right. The war pitted the fascist right, led by Nazi Germany, Fascist Italy, and Imperial Japan, against the Allied powers, which included both right-leaning capitalist democracies like the United States and left-leaning communist states like the Soviet Union. Although the right-wing fascists were defeated, the ideological clash between capitalism and communism didn’t end with the war.

The Cold War (1947–1991) that followed World War II was dominated by the left-right ideological battle on a global scale. The left, led by the Soviet Union, sought to spread communism, advocating for state control of the economy, the abolition of private property, and a classless society. The right, led by the United States and its Western allies, championed capitalism, individual freedoms, and democratic governance. The Cold War saw proxy conflicts fought in countries like Korea, Vietnam, and Afghanistan, as the two ideologies vied for dominance.

During this period, the right wing in Western democracies, particularly in the United States and Western Europe, became strongly associated with free-market capitalism and opposition to communism. Conservative leaders, such as Margaret Thatcher in the UK and Ronald Reagan in the U.S., advocated for deregulation, lower taxes, and a reduction in the role of government in economic affairs. They portrayed their policies as a defense against the perceived threats of left-wing socialism and communism.

On the left, many socialist and social democratic parties in Europe distanced themselves from Soviet-style communism, instead advocating for a mixed economy where the state would provide social welfare programs, healthcare, and education while still maintaining capitalist markets. This form of “social democracy” took root in countries like Sweden, Norway, and Denmark, where strong welfare states and capitalist economies coexisted.

Post-Cold War: New Conflicts and Populism

After the Cold War, the collapse of the Soviet Union marked the triumph of capitalism and liberal democracy, but the ideological battle between left and right was far from over. In the 1990s, many left-leaning parties in Western countries shifted toward the center, adopting what became known as the “Third Way.” Leaders like Bill Clinton in the U.S. and Tony Blair in the UK embraced free-market economics while maintaining social safety nets, merging aspects of both left and right ideologies.

Meanwhile, neoliberalism—characterized by deregulation, free trade, and globalization—gained momentum, particularly among right-wing parties. However, this economic liberalization led to growing inequality, discontent among the working class, and an erosion of traditional industries. As a result, populist movements began to rise in both the left and right.

Image Source: Post-Brexit EU-UK

On the right, populist leaders capitalized on frustrations with globalization, immigration, and perceived political elites. Figures like Donald Trump in the U.S. and the Brexit movement in the UK emphasized nationalism, promising to protect national sovereignty and return jobs to domestic workers. Meanwhile, on the left, movements like Occupy Wall Street and figures such as Bernie Sanders pushed back against corporate power, income inequality, and the influence of the wealthy elite, arguing for more extensive government intervention in the economy. The rise of populism and increasing polarization has also redefined the political landscape. The right continues to advocate for economic deregulation and nationalism, while the left calls for expanded social programs and environmental protections.

Conclusion: The Modern Left, Right, and Beyond

Image Source: Reddit -> Very Details Political Compass

The terms “left” and “right,” born from the French Revolution, continue to shape the world today, but they no longer define the full complexity of political ideologies. While the left pushes for progressive reforms, as exemplified by Obama’s “Change,” and the right advocates for a return to tradition, as in Trump’s “Make America Great Again,” politics is far more nuanced. Centrism, libertarianism, social democracy, and environmentalism are just a few examples of the many ideologies people turn to in shaping their vision for society. Some experts argue that the traditional definitions of right and left make little sense in today’s landscape. This Hidden Forces podcast discussion with Henry Olsen a columnist that provides commentary on American politics is an excellent example of such discussion. Understanding these diverse ideas as we navigate modern politics can help us better grasp the complexity of today’s political landscape.

Finally, in the upcoming 2024 Presidential Elections, the right-wing candidate, Donald Trump, is running on a platform that emphasizes securing the U.S. border, deporting illegal immigrants, and boosting domestic manufacturing to make America more self-reliant. He focuses on lowering taxes, ending inflation, and positioning the U.S. as the world’s dominant energy producer. Trump advocates for strengthening the military, protecting Social Security without cuts, and combating what he refers to as “radical gender ideology” in schools. He also supports secure elections through measures like voter ID and paper ballots. Overall, his platform promotes strong national defense, economic independence, and a focus on traditional values and freedoms. You can read more about Trump’s platform here. On the other hand, the left-wing candidate, Kamala Harris, is running on the Democratic Party platform that revolves around building economic growth for working families, expanding access to affordable healthcare, and tackling the climate crisis through clean energy investments. It prioritizes social justice by defending reproductive rights, addressing racial and gender inequities, and protecting LGBTQ+ rights. The platform also calls for stronger gun safety laws, law enforcement reform, and greater access to education, including lowering student debt and expanding universal pre-K and affordable childcare. Ultimately, it seeks to promote equity, environmental sustainability, and comprehensive social reforms. You can read the DNC Platform for 2024 here.

Bonus: The Story Behind Red and Blue in U.S. Politics and the Origins of the Donkey and Elephant Symbols

Interestingly, the association of red and blue with U.S. political parties is a relatively recent development. Before the 2000 U.S. Presidential election, there was no consistent color scheme for Democrats and Republicans. In fact, throughout much of the 20th century, news outlets used varying color assignments, often reversing what we know today.

The 2000 election, a prolonged battle over the results between George W. Bush and Al Gore, solidified the red and blue divide. Major TV networks consistently used blue for Democrats and red for Republicans in their electoral maps, and the color scheme stuck. This is somewhat unusual compared to other countries, where red signifies left-wing or socialist parties, and blue is often associated with conservative movements. Nonetheless, in the U.S., blue has come to symbolize the Democratic Party and red the Republican Party—creating a lasting visual shorthand for America’s political divide. These videos provide some useful context into this history: Why red means Republican and blue means Democrat, America 101: Why Red for Republicans and Blue for Democrats? | History, and Why Red is Republican & Blue is Democrat: States & Party Colors | Democrat & Republican History.

Image Source: Why Democrats are donkeys and Republicans are elephants

As for the donkey and elephant, these iconic symbols have deeper historical roots. The donkey first became associated with the Democratic Party in the 1828 presidential campaign of Andrew Jackson, when his opponents called him a “jackass.” Instead of shying away from the insult, Jackson embraced the image, and it stuck. The elephant came to symbolize the Republican Party thanks to a political cartoon by Thomas Nast published in Harper’s Weekly in 1874. The cartoon depicted the Republican vote as an elephant, solidifying its association with the party over time. You can learn more in this video: Two Minute Explainer: Why are Republicans elephants and Democrats donkeys?.

References

While I have cited several sources in this blog post, below are some other resources that I used to write it and that can be useful for your further learning.

• The Political Spectrum Explained In 4 Minutes

• What to Know About the Origins of ‘Left’ and ‘Right’ in Politics, From the French Revolution to the 2020 Presidential Race

• Understanding the political spectrum

• Otto von Bismarck

• What’s the origin of the right and left terms in politics?

• Thomas Nast’s first Republican elephant print

• Political Spectrum Explained | What is the Political Spectrum? | Left Vs Right

• Political Spectrums Explained — Why is there a left-wing and right wing?

• What’s the deal with left and right in politics? | Politics Explained (Easily) | ABC News

• Every Political Ideology Explained in 8 Minutes

• Political Ideology: Crash Course Government and Politics #35

• The Political Spectrum

• How we know what Left and Right actually mean: who’s who, 1789-1917

• Where Did the Terms ‘Left Wing’ and ‘Right Wing’ Come From?

• The French Revolution: Crash Course World History #29

• Left Wing, Centre and Right Wing Explained | Conservatives, Labour & Liberal Democrats Summarised!

• The Three Estates

• A history (and defense) of left vs right

• 2024 Democratic Party Platform

• Can You Guess How Americans Feel About Harris’s Platform?

• The Napoleonic Wars - OverSimplified (Part 1)

• The Napoleonic Wars - OverSimplified (Part 2)

• The Global Reach of the Napoleonic Wars

• THE INDUSTRIAL REVOLUTION IN EUROPE

• Talking About Politics: LEFT WING & RIGHT WING

Image Source: Cartoon Movement | Gatis Sluka

I’ll start this post with a light-hearted apology to my dear friends and other readers who, upon seeing the title, are already rolling their eyes. “Soccer? What do you mean? It’s football!” I hear your cries and trust me; I get it. After all, I grew up in Nigeria, where the word “football” is as sacred as jollof rice at a family gathering. But bear with me—I have my reasons. Back in my secondary school days, we played a lot of football—or, as we liked to call it, soccer. Yes, you read that right. We were the kids who thought we were too cool for “football.” I mean, we even refused to call akara or kosai by the names everyone else used, insisting on the far more sophisticated “Bean Cake.” So, when my friends wanted to hit the pitch, they didn’t say, “Let’s go play some football.” No way! They’d say, “Let’s go tap soccer!” It was a whole vibe. So, for nostalgia’s sake—and maybe because I now live in America, where “soccer” reigns supreme—I’ll be sticking with “soccer” for this post. But rest assured, my football-loving heart is still very much intact.

Now, onto the main event: why am I writing about tactics in soccer? I’m no tactical genius, but I’ve been playing in a Division 3 league, and let’s say I’m itching for more wins. We’ve got some incredible folks on our team, but when we stack up against other squads, the odds aren’t always in our favor. So, what’s a desperate player to do? Dive headfirst into the world of soccer tactics, of course! I’ve been reading, researching, and obsessing over formations, marking strategies, and every other tactical tidbit I can find to help my team squeeze out that extra edge and, hopefully, leave the other teams in the dust. It also helps that, at various times, our team captain and coach have referred to me as the Assistant Coach (though sometimes I’m just the assistant to the assistant coach—but you get the picture). My team has all the right personnel; hence, I am optimistic that slight adjustments to playing style and approach will propel us to several victories. So, should we go all-in with a high-intensity gegenpressing strategy (that’ll be hard to keep up)? Stick to long balls? Or maybe build up from the back like the pros do? And what about formations—4-4-2, 4-3-3, 4-2-3-1? The options are endless!

As I write this, I’m also reminded of my final year in secondary school, when I spent countless hours playing Football Manager with my close friend, Bamshak, and my roommate, Dama. We would obsess over tactics, trying to determine how to gain an advantage. Admittedly, much of the game was about playing the market to build the right squad, but how you put that squad together, and the game plan mattered just as much. For old-time’s sake, below is a picture of me and Bamshak. You can probably guess what we were discussing. Let me help you—we were playing Football Manager on a Saturday (we went to boarding school) when we lost power (in Nigeria, frequent power outages were the norm). So, we finally decided to step outside and ended up sitting by the basketball court, discussing tactics and how to maximize our chances of winning. What a wonderful time that was.

As I’ve already said, I’m no expert, but I’m learning as I go, and I want to share some of what I’ve uncovered. In this post, I’ll dive into tactical divisions of the pitch, phases of play, positional play, marking strategies, and a few other exciting things I have learned about tactics in soccer. So, stick around, enjoy the read, and share your thoughts on different vital tactics and considerations in soccer. And as an excellent introduction to the positions players take on the pitch, check out this video as well as this one to get a quick overview of formations.

Basic Soccer Pitch Zones

Image Source: Scientific Reports Journal

The soccer pitch is commonly divided into zones to better understand and strategize play. The two most basic ways to divide the pitch are the 4-zone division, with dividing lines running from sideline to sideline, and the 5-channel division, with dividing lines running between the end lines.

4-Zone Division

The 4-zone division mirrors the progression of play, starting from the goalkeeper and advancing toward the opposition’s goal. The zones are as follows: Starting (Build) Zone, Progression Zone, Unbalanced (Penetrate) Zone, and Goal (Finish) Zone. These zones aren’t fixed to specific areas of the pitch but shift depending on the opposition’s shape. Typically, each zone begins just behind the opposition’s line of defense.

This division ties into the “phases of play” in soccer, which include in possession, out of possession, transitions, and set-pieces. I’ll focus on the in-possession phase here. It usually starts from the back, with the team in possession occupying the Build Zone. The team pushes the ball up the pitch, often at a high tempo with quick, short passes, offering support to the ball carrier. After bypassing the opposition’s first line of press (where the opposition applies immediate pressure to regain possession), the team enters the Progression Zone, attempting to break through the opposition’s block and move toward the Goal Zone. The Progression and Unbalanced Zones are often considered a single unit, making up the middle third of the pitch. In these zones, the team aims to advance into the final third, usually sending the ball to wingers or strikers in the Unbalanced Zone to create scoring opportunities in the Goal Zone. Some tactics might skip the first two zones entirely, with the goalkeeper sending long balls directly into the Unbalanced and Goal Zones—a common strategy in long-ball tactics. Teams employ various techniques to advance through these zones, such as forming triangles to create clear passing options.

5-Channel Division

The 5-channel division divides the pitch into the wings (outside right and left lanes), half-spaces (inside right and left lanes), and the center lane. Typically, one attacking player occupies each channel, maintaining a presence across the pitch. The players at the back use this structure to advance the ball forward, creating opportunities to score by exploiting the channels.

Other Division Methods

Beyond these basic methods, more sophisticated ways to divide a soccer pitch exist. One such method is the 18-zone division of the pitch, as shown in the image below:

Image Source: Frontiers of Psychology

This more detailed division helps players better understand their roles and positions, allowing them to adapt based on the opposition’s strategy. Ideally, no more than two players should occupy the same zone simultaneously to cover the pitch’s full depth and width. Zone 14 is considered the most dangerous in this division because most through balls and critical passes tend to originate from here. Controlling this zone has been proven to increase the chances of scoring goals. It is considered a vital distribution zone and the prime spot for sending balls into zone 17, where most goals are scored from. You can learn more about the importance of zone 14 in these articles: What is “Zone 14” in football?, How to create chances through Zone 14 and this video: Why is Zone 14 Important in Football?.

Even more intricate is Pep Guardiola’s 20-zone system, which optimizes for positional play and ensures that every player knows where they need to be and when. While I won’t delve deeply into this system here (a few words cannot do justice to the zoning strategies used by the soccer genius Pep Guardiola), here are some excellent resources to explore: How Pep Sees Football Differently | What Separates Pep Guardiola | Pep Guardiola’s 20 Zone System, Vulnerable channels and 20 zones: the tactics behind Guardiola’s title win. Even in Pep’s tactics, zone 14 remains crucial, offering midfielders the best opportunities to generate goals.

Now that we understand how the pitch can be divided let’s explore the phases of play in a soccer game.

Phases of Play

The phases of play in soccer can be divided into four main categories: In Possession (Attack), Out of Possession (Defense), Transitions, and Set Pieces. When a team is in the In Possession phase, they progress through the pitch, starting with the build-up from the back, moving into the progression phase through the middle, and finally reaching the attack phase in the final third of the pitch. In this final stage, the team focuses on creating and executing goal-scoring opportunities.

For each of these three In Possession stages, there is a corresponding Out of Possession stage that mirrors it, defined by the ball’s location on the pitch. A team may be in a high block stage when pressing the opposing team as they attempt to build up play. In the high block stage, the team maintains a high defensive line and a compact shape to win the ball back. If the pressing line is broken, the team reshapes and falls back into a mid-block stage, where the defensive line stays relatively high to prevent the opposition from advancing toward their goal. Finally, in the low block stage, the team adopts a more intensive and deliberate defensive approach, maintaining a compact shape to prevent the opposition from making dangerous attempts near the goal.

Teams are generally either In Possession or Out of Possession, and the moments when the ball changes hands constitute the Transitions phase. This can be an out-to-in transition (defense to attack) or an in-to-out transition (attack to defense). These moments of transition are very brief but crucial. If a team loses the ball, they must quickly decide whether to initiate a counter-press to win it back immediately or to delay the opposition’s ball carrier while rapidly falling back into a defensive shape. Conversely, the team that has just regained possession must decide whether to counterattack immediately if the opposition is unbalanced or to transition the ball out of pressure to reestablish possession and build up to an attack.

Finally, the Set Pieces phase occurs during free kicks, corner kicks, penalties, throw-ins, and sometimes kickoffs. Every attacking set piece has a corresponding defensive setup. Set pieces are critical moments in the game, as they determine how a team positions itself and prepares to defend or take control of the ball to launch an attack.

Optimizing Tactics: Some Strategies and Methods

Now that we have a basic understanding of pitch divisions and the phases of play in a soccer game, we can dive into some intriguing tactical nuggets that teams use to optimize their chances on the pitch. First, we’ll explore some defensive strategies before moving on to broader team tactics that shape the game.

Defending Tactics: Keeping the Opposition at Bay

Man vs. Zonal Marking

Defending is an art in soccer, and marking is one of its most crucial elements. Teams often choose between man and zonal marking depending on their defensive philosophy. In man marking, defenders are tasked with shadowing specific opponents, ensuring that the opposing team’s key players are tightly marked. This can be incredibly effective against star players who can turn the game on its head with a single moment of brilliance. On the other hand, zonal marking focuses on covering areas of the pitch rather than individual players. This approach helps maintain the team’s shape and prevents gaps from opening up in the defense. A great example of zonal marking can be found in the Italian team’s famous catenaccio system, which is built around a tight, organized defense that prioritizes space over players. Both systems have their merits, and teams often use a combination of the two to adapt to different situations on the pitch. While reading and writing this post, I stumbled across various resources, particularly about the Italian catenaccio system. You can watch this video to see Italy’s beautiful defensive organization and movement in Euro 2016.

High Defensive Line

Another critical defensive tactic, especially in modern soccer, is playing with a high defensive line. This strategy involves pushing the defense further up the pitch, effectively compressing the space available for the opposition to play in. It’s a hallmark of teams that like to dominate possession and press aggressively, like Jürgen Klopp’s Liverpool or Pep Guardiola’s Manchester City. Learn about their tactics in this video. The high line is a double-edged sword; it can suffocate an opponent’s build-up play, but it also leaves space behind that can be exploited by quick forwards. One spectacular failure of this tactic is in this poor-quality but helpful video of Borussia Dortmund taking advantage of FC Bayern Munich’s incredibly high defensive line.

Parking the Bus

At the opposite end of the spectrum, we have the infamous tactic of parking the bus. This ultra-defensive approach involves setting up deep in one’s own half, with many players behind the ball to frustrate the opposition. José Mourinho’s Chelsea utilized this tactic at various times in their title-winning 2014-15 Premier League season, maintaining a low block defensive position and preventing opposing teams from penetrating. You can learn about their tactics in this video or this analysis of a Chelsea vs Barcelona game.

Team Tactics: Shaping the Game

Total Football

Now, let’s move on to broader team tactics that have revolutionized the game. Total Football, pioneered by the Dutch in the 1970s, is all about fluidity and versatility (for instance, in the 1974 World Cup). In this system, players are not confined to specific positions. Instead, they move freely across the pitch, with the understanding that another player will cover their position. This requires immense technical skill and tactical awareness. Johan Cruyff’s Ajax and the Dutch national team were the epitome of Total Football, with players like Ruud Krol and Johan Neeskens seamlessly switching roles to create a dynamic and unpredictable attack. Fictionally, AFC Richmond deployed this tactic in the Apple TV+ Series: Ted Lasso (strongly recommend).

Positional Play

In contrast to the fluidity of Total Football, positional play—often associated with Pep Guardiola—emphasizes structure and discipline. Players are expected to occupy specific zones on the pitch, ensuring the team maintains its shape and effectively controls space. This approach allows for better ball circulation and quick pressing when possession is lost. Guardiola’s Manchester City squad have become masters of positional play, using their precise passing and spatial awareness to dominate games and create scoring opportunities.

Tiki-Taka

Building on the principles of positional play, tiki-taka is a style that prioritizes short, quick passes and movement to keep possession and patiently break down the opposition’s defense. Popularized by Pep Guardiola’s Barcelona and later adopted by the Spanish national team, tiki-taka is a mesmerizing blend of precision and patience. The 2010 World Cup-winning Spanish team is perhaps the best example of tiki-taka in action, where players like Xavi, Iniesta, and Busquets dictated the tempo, slowly but surely wearing down their opponents before delivering the killer blow. Learn more about it in this video.

Build-Up Play

Central to many of these tactics is the concept of build-up play. This involves carefully moving the ball from the back through the midfield and into the attack, often through a series of short, controlled passes. Build-up play is designed to maintain possession and create openings in the opposition’s defense. It’s a systematic approach, contrasting with the more direct styles that rely on long balls or quick transitions. Teams like Manchester City under Guardiola or Barcelona under Guardiola and Luis Enrique have perfected build-up play, using it to draw opponents out of position before striking with lethal efficiency.

Gegenpressing

Finally, we come to gegenpressing, a tactic that combines intense pressing with quick transitions. When a team loses possession, they immediately press the opposition to win the ball back, often in advanced positions. This strategy, championed by Jürgen Klopp at Borussia Dortmund and Liverpool, is about quickly regaining control and catching the opponent off guard. The 2019 Champions League semi-final between Liverpool and Barcelona is a masterclass in gegenpressing, with Liverpool’s relentless pressure forcing Barcelona into mistakes and leading to one of the most dramatic comebacks in football history. See an analysis of that game here.

Each tactic has its own place in the rich tapestry of soccer strategy. Whether it’s the fluid dynamism of Total Football, the meticulous precision of tiki-taka, or the relentless intensity of gegenpressing, these methods help teams optimize their performance and outmaneuver their opponents on the pitch. Understanding these strategies enhances our appreciation of the game and provides valuable insights into how the world’s best teams consistently find ways to win.

Closing

Writing this post has been an absolute joy—I’ve loved diving deep into the tactics that make soccer so fascinating. It’s been a rewarding journey, from revisiting old memories to discovering new strategies. I focused on the parts I found most interesting, but there’s so much more to explore. I hope you keep digging deeper into this beautiful game.

Thanks for reading! I’d love to hear your thoughts and any tactics that have caught your eye. Until next time, keep enjoying the beautiful game! ⚽

References

And resources to learn more from:

• Formations: football tactics explained

• Man Marking or Zonal? Outlining the Arguments for Both Defensive Approaches

• What Is Catenaccio?

• Key Tactical Decisions That Took Chelsea to the Premier League Title

• Klopp’s Tactics Behind The Greatest Season | Liverpool’s 2019/20 Tactics | How Klopp Improved LFC |

• Formations Explained

• Tifo Football Encyclopedia

• Tactics Explained | Tifo Football

• Fundamentals of Football

• Formation Principles

• Player Principles

Image Source: wizard zines

Introduction

You might have started reading this blog post with some curiosity. For one, the title might have left more questions than answers unless you’re well-versed in big data analytics or distributed systems. Let’s begin by introducing the concept of distributed systems in computing. From there, we’ll explore broadcasting in these systems and finally dive into the specifics of broadcasting in Spark. I hope you enjoy the journey.

Distributed Computing Systems

Imagine you love to cook, and your friends always rave about your dishes. Encouraged by their compliments, you decide to open your restaurant. Initially, you’re the only chef handling everything yourself. But as word spreads, business booms and you quickly realize you can’t manage it alone. You hire more chefs and expand your kitchen to keep up with demand.

Now, on a busy evening, your kitchen is abuzz with activity. Each chef works at a different station—one handling appetizers, another on main courses, and so on. Together, they ensure that every dish is prepared perfectly and served on time. This setup is much like a distributed system in computing:

• Chefs as Nodes: Each chef in your kitchen is like a node in a distributed system, working independently yet contributing to the same larger goal.

• Division of Labor/Stations as Subsystems: Different dishes are prepared at different stations, just as tasks in a distributed system are divided among various nodes.

• Coordination: As the head chef, you ensure that all stations work in sync, much like how a distributed system coordinates the work of its nodes to produce the final output.

• Parallel Processing: Multiple dishes are prepared simultaneously at different stations, just as tasks are processed in parallel across multiple nodes in a distributed system, improving speed and efficiency.

• Fault Tolerance: If a chef burns a dish, the kitchen adapts without disrupting the entire operation. Similarly, a distributed system continues to function smoothly even if one or more nodes fail.

Just as your restaurant thrives by distributing the workload across multiple chefs, distributed computing systems efficiently handle large tasks by dividing them across multiple computers. The benefits of distributed computing are tremendous, many of which you can read about in this AWS Article, this IBM Article, or this Geeks for Geeks Post. Now, onto broadcasting.

Broadcasting (As it applies to Distributed Computing)

After several months of running your restaurant smoothly with a trusted team of chefs, you take a well-deserved break. Your travels take you to Europe, where one of your stops is the historic Bletchley Park in Milton Keynes, renowned for its pivotal role in World War II. Bletchley Park was where brilliant minds, including Sir Alan Turing, Gordon Welchman, Joan Clarke, and many others, worked tirelessly to crack the German Enigma code. The complexity of these codes meant that a single person, or even a small team, couldn’t break them quickly enough to be effective. So, the Government Code & Cypher School (GC&CS) assembled a large team of experts, creating what was effectively a distributed system.

At Bletchley Park, each codebreaker worked simultaneously on different parts of the problem, much like nodes in a distributed computing network. But before they could begin their work, they needed to be equipped with critical information—cipher keys, known plaintexts, and the latest intelligence. This process of equipping every codebreaker with the necessary data is akin to broadcasting in distributed computing. Once they had this information, they could work independently, much like how nodes process tasks in parallel. The distributed system at Bletchley Park, powered by the efficient broadcasting of information, was crucial in breaking the Enigma code and contributing significantly to the Allied victory.

In distributed computing, broadcasting refers to the process of sending a piece of data to all nodes in a cluster, allowing them to use the data locally without requiring repeated network communication. This approach minimizes communication overhead and enables more efficient computation, much like how the codebreakers at Bletchley Park worked more effectively with the information they were given.

Broadcasting in Spark

Apache Spark is a powerful open-source distributed computing system designed for fast data processing on large-scale datasets. It is widely used in big data analytics because it performs complex computations quickly and efficiently across a cluster of computers. Spark builds upon the Hadoop ecosystem and can work seamlessly with the Hadoop Distributed File System (HDFS). HDFS is a distributed storage system that allows Spark to store and access large datasets across multiple nodes in a cluster. This distributed storage capability is crucial for handling massive volumes of data, enabling Spark to process them in parallel across many machines.

However, when working with large datasets and complex computations, efficient data sharing across the nodes becomes essential. This is where the concept of broadcasting comes into play. In Spark, broadcasting is a mechanism that allows a small dataset or a piece of data to be efficiently shared with all the nodes in the cluster. Instead of sending the same data multiple times to different nodes, Spark broadcasts the data once to all nodes. Each node then stores a local copy of the broadcasted data, allowing it to access the data quickly without repeatedly fetching it from the driver or another central location. Let’s consider join operations in Spark as an example of how this works.

Shuffling: The Default Approach for Joins

In distributed computing, particularly in Spark, one of the critical operations is joining datasets. Spark uses a technique called shuffling when performing certain operations on datasets, such as joins, grouping, and aggregations. Shuffling redistributes the data across different nodes in a Spark cluster, grouping data elements that need to be processed together on the same node. For instance, in a join operation, shuffling ensures that matching keys from different datasets are brought together in the same partition, allowing the join to be executed.

While shuffling is effective in ensuring that data is correctly aligned for operations like joins, it comes with significant downsides, mainly when dealing with large datasets:

• Increased Network Traffic: Shuffling involves redistributing data across the network, which can substantially increase network traffic. This can be particularly problematic for large datasets, where the volume of data being transferred can become a bottleneck.

• High I/O: As data is shuffled, it often needs to be written to and read from the disk, increasing disk I/O operations. This further slows the computation, especially if the disks are not fast enough to handle the load.

• Resource Intensity: Shuffling is resource-intensive, requiring significant computational power and memory. This can lead to longer processing times and increased costs, especially in cloud environments where resources are billed based on usage.

Broadcasting: A More Efficient Alternative

To mitigate the downsides of shuffling, especially for join operations involving a large dataset and a smaller one, Spark employs the concept of broadcasting. Broadcasting is particularly useful when one of the datasets in a join operation is small enough to fit into the memory of each node. Instead of shuffling both datasets to bring matching keys together, Spark broadcasts the smaller dataset to all nodes in the cluster. Here’s how it works:

Imagine Spark needs to join a large dataset with a much smaller one. The large dataset is already distributed across the cluster's nodes. If Spark used shuffling, it would have to redistribute the data in large and small datasets, grouping matching entries together. This would involve significant network transfer and could be highly inefficient.

Instead, with broadcasting, Spark sends a single copy of the smaller dataset to each node. Each node then performs the join locally, using its portion of the large dataset and the broadcasted small dataset. This approach yields advantages in reduced network traffic (large amounts of data are not shuffled across the network; only the small dataset is broadcasted, resulting in a reduction in the amount of data that needs to be transferred), faster computation (as each node can perform the join operation locally without needing to wait for data from other nodes and without the need for extension disk I/O, as the data is already in memory), efficient resource utilization (as the need for shuffling is eliminated, broadcasting reduces the strain on computational and memory resources; reducing the overall cost of running Spark jobs).

More on Broadcasting in Spark

It is vital to appropriately leverage broadcasting to truly make Spark computations more efficient. When performing operations such as joins or aggregations, where a small dataset needs to be applied to a large one, using Spark’s Broadcast variables can significantly improve performance. Spark automatically distributes these broadcast variables to all the nodes, which remain cached in memory on each node, allowing for fast access. Developers can create broadcast variables using the SparkContext.broadcast() method, ensuring the data is distributed efficiently across the cluster. You can learn more about using broadcasting in spark operations with these articles: Broadcast variables in spark work like they sound, Understanding Broadcast Variables In Apache Spark.

Automatic Broadcasts and the Spark Broadcast Threshold in Spark

Spark further simplifies the use of broadcasting by automatically determining when to broadcast small datasets during certain operations like joins. By default, Spark automatically broadcasts any dataset that is smaller than a predefined threshold, known as the broadcast threshold. This threshold is controlled by the configuration parameter spark.sql.autoBroadcastJoinThreshold, which is set to 10 MB by default (and maxes out at 8GB). When a dataset falls below this size, Spark will automatically broadcast it to all nodes in the cluster without requiring explicit instructions from the developer. This automatic broadcasting is particularly useful because it optimizes performance without the need for manual intervention, allowing Spark to handle joins more efficiently by avoiding unnecessary shuffling.

Despite the performance improvements from broadcasting, it’s essential to be mindful of the broadcast threshold and how broadcasting is used in general. While broadcasting small datasets can significantly improve performance, attempting to broadcast datasets that are too large can lead to memory issues, as each node must store the entire broadcasted dataset in memory. If a dataset exceeds the broadcast threshold and is still small enough to benefit from broadcasting, the threshold can be adjusted by setting the spark.sql.autoBroadcastJoinThreshold parameter to a higher value. Conversely, if broadcasting is not desirable for certain operations, this threshold can be lowered to prevent automatic broadcasts. Understanding and tuning this threshold allows developers to strike the right balance between leveraging Spark’s automatic optimizations and effectively managing the resources available in their cluster.

Conclusion

Broadcasting in Spark is a powerful tool for optimizing distributed computations, mainly when dealing with large datasets and complex operations like joins. By minimizing network traffic and reducing the need for shuffling, broadcasting can significantly improve the efficiency and speed of your Spark applications. Understanding when and how to use broadcasting and tuning the broadcast threshold is crucial for maximizing the performance of your Spark jobs. With these insights, you can make more informed decisions that enhance the effectiveness of your distributed data processing.

References

And resources to learn more from:

• The Women of Bletchley Park

• Class Broadcast<T>

• Broadcast Variables

• Video on Spark Broadcast Variables

• Coding Example of Broadcasting in PySpark

• Spark Performance Tuning

• Broadcast join in Spark (including how to configure Spark auto broadcast)

• Usage of “Broadcast Variable” in “Joins” in “Apache Spark”

• Join Strategies in Apache Spark— Deep Dive

• Primer on Spark Join strategy

Image Source: XKCD

Accompanying Spotify Playlist (because “Traffic Jams”)

“Why, why? But why is this happening?” This was my exact thought when I landed at the Seattle airport on a seemingly ordinary Wednesday afternoon. My maps app lit up with bright red and yellow lines, showing the major highways into the city—I-5, I-190, SR 520—choked with traffic. I wasn’t thrilled, to say the least. I knew I could take alternate routes like Highway 99 or 509, but that didn’t quell my burning curiosity. It also didn’t help that on my flight over; I read a Komo News report about how INRIX, the location and traffic data analytics company, found that the average Seattle driver lost 58 hours in traffic jams in 2023, making the city home to the 27th-worst traffic in the world (you can read about the findings and methods here). So why was there so much traffic? It couldn’t be a car crash; what were the odds of massive pile-ups on three highways on a sunny Wednesday afternoon? Road work didn’t seem likely either, as that typically happens at night or on weekends. And it wasn’t rush hour. So, what was causing the gridlock? I wanted to find out.

Typically, traffic jams occur when sections of the roadway are blocked off, or obstructions on the road require drivers to use alternate routes. For instance, when there is a car accident, traffic might be rerouted, or when there is construction work, cars are diverted to different routes. Other instances include inclement weather, requiring drivers to proceed cautiously; rush hour, where there is a sudden and massive increase in the number of cars on the road; or saturation, which in traffic engineering means the number of vehicles on the road has exceeded its capacity, causing a jam to build up. However, while these are reasonable explanations for traffic jams, they don’t quite explain instances where there is a traffic jam with no apparent cause. Or cases where a jam builds up and dissipates without a clear reason for its start or end. This brings us to the concept of phantom traffic.

To introduce phantom traffic, I will start by sharing a recent conversation with my skip manager at work. He talked about participating in STP, a 200+ mile bike ride from Seattle to Portland. This multi-day cycling event draws thousands of riders willing to brave the challenge. You can learn more about STP here. Now, back to my skip manager. Because this is a long event, conserving energy and using all available advantages is crucial. Participants take advantage of the draft effect, where a rider in front forces through the air, creating a wake or slipstream that allows riders behind to move at the same speed with much less effort. However, this process is complicated, as my skip manager highlighted.

If a rider in front needs to brake, simply braking would endanger the riders behind, following closely at the same speed. Sudden braking can take them by surprise, cause an immediate crash, or cause them to react suddenly in a manner that endangers their and other’s safety. To avoid this, the rider in front communicates the need to brake to the person behind, who then passes the message down the line. This ensures that everyone is prepared, and the rider in front can brake while each rider behind reacts accordingly without running into anyone. For this system to work, all riders must be in sync, moving and slowing down or speeding up at the same rate.

We see a similar effect when driving on a highway with several cars moving at roughly the same speed. If all drivers maintain their speed and roughly the same spacing from each other, traffic flows steadily. But say one driver abruptly slows down (even if just slightly); each driver behind notices and reacts, but they brake a little more strongly as they were not fully prepared for the event. This creates a chain reaction, with a wave of slowing vehicles propagating through the roadway. Vehicles slow down to avoid a collision, and when situations are safe, they accelerate back up to normal speed. This causes a traffic jam, and vehicles enter the jam faster than vehicles can accelerate out of it; hence, the jam continues until the rate at which vehicles enter the jam becomes low enough for the vehicles leaving the jam to restore normal flow. And as human behavior is imperfect, these traffic jams will continue as we utilize our roadways.

Typically, this effect is not observed on roads where the density of cars is low. In those cases, cars can quickly adjust to abrupt changes in speed and maintain a smooth traffic flow. However, as the number of cars increases to a critical mass, we start to see this effect and the corresponding stop-and-go traffic jams.

To model this effect, I took inspiration from the work done by researchers and people who have studied traffic, and I built a phantom traffic simulator to accompany the blog post. You can find it in this Github Repository. You can fork the repo and play around with the simulator, tweaking it, perhaps to see how things change when you increase the number of vehicles, modify how the speed changes, etc. I have tried to keep it lightweight, but if you can improve it and make it more robust, I would love to see your work. :)

You can also find other great traffic simulators here or see how traffic flows through different types of junctions this video.

Beyond increasing the time it takes for road users to get to their destinations, phantom traffic is also very inefficient for vehicles because when cars are decelerating or coasting, they burn less fuel and emit much less toxic gases. But when cars drive and stop, and then drive and stop in areas with phantom traffic, each accelerating run burns more gas and emits more pollutants. Research has also shown that phantom traffic causes more accidents as drivers could overreact or underreact to what might happen in front of them (think back on the biking example from STP). Optimistically, autonomous vehicles driving on the roadway dynamically adjust their average speed with knowledge of the traffic ahead, which could help reduce phantom jams. Likewise, such driverless vehicles can be modeled to compute the best following distance and speeds to prevent traffic buildup or react in ways that reduce them.

In conclusion, traffic that seemingly comes out of nowhere is caused by humans being imperfect, and our actions on the road affect other drivers, which can lead to the buildup of jams. Or, as put more bluntly in this Freakonomics article: What Causes Traffic Jams? You. With this knowledge, we can be more forgiving and calm when traffic seems to appear out of nowhere, and more conscious of how our driving (say slowing down slightly to go back 30 seconds on a podcast), can affect other drivers and the flow of traffic on the road.

As a side note, I’ve also been reading a little about why increasing the size of the road does not solve problems with traffic jams. Here are some great articles to learn more about it: What’s Up With That: Building Bigger Roads Actually Makes Traffic Worse, Question Your World: How Does Roadway Expansion Cause More Traffic?, as well as a video.

References

And resources to learn more from:

• Why the @#$% is there so much traffic? - Benjamin Seibold

• Shockwave traffic jam recreated for first time

• Traffic Waves

• Traffic Jams

• A Mathematical Introduction to Traffic Flow Theory

• Traffic Modeling - Phantom Traffic Jams and Traveling Jamitons

Image Source: XKCD

Introduction

In computing, permissions define the level of access, privilege, and control that users have over files, applications, and other system resources. These permissions are divided into three broad categories: Read (R): letting users view the content of files and directories, Write (W): letting users create, modify, and delete files and directories, and Execute (X): letting users run files or programs.

Unauthorized access to system resources, especially by malicious actors, can lead to dire consequences. Therefore, it is important to set and manage permissions appropriately. In this post, I’ll share briefly about how to manage permissions in Linux.

Users and Groups

To begin, we must first understand the concept of users and groups in Linux. A user is an entity that can log in and interact with the system, typically representing a person or a service. Each user has a unique username for friendly identification, a unique numerical identifier (User ID - UID) for the system to identify the user, and a home directory (usually in the form /home/username) to store user-specific files.

A user can be:

• The Root User or Superuser, with unrestricted access to all commands and files on the system, allowing them to perform any actions and system-level tasks.

• A regular user, with limited permissions and restrictions to their own files and certain system resources.

• A System User, associated with system services and applications used to run background processes to keep the system functioning.

Each user’s information, including their username, UID, home directory, and shell, is stored in the \/etc\/passwd file, while passwords are typically stored in a hashed format in the \/etc\/shadow file.

Groups, on the other hand, represent collections of users that can be managed as a single unit. They are used to manage user permissions and access control for files, directories, and system resources. Each group has a Group ID (GID) and is defined in the \/etc\/group file, which lists group names, GIDs, and the users who are members of the group.

Each user is assigned to a primary group. The Primary Group is specified in the user’s account information and determines the group assigned to files created by the user. Each user must belong to one primary group. Secondary Groups specify one or more groups to which a user can belong, granting additional permissions and access to resources shared among users in the group. Typically, a Linux user can belong to up to 15 secondary groups.

Users and groups together are used by the system to define and enforce access and permissions.

Permissions

As mentioned earlier, each file and directory in a Linux system has three types of permissions: r (read), w (write), and x (execute). These permissions are defined for three types of entities: the file/directory owner (user), the group, and others (everyone else in the system). Permissions are represented as a 10-character string, such as -rwxr-xr--.

The first character indicates the file type (- for a regular file, d for a directory, and l for a symbolic link), and the remaining nine characters represent the permissions for the owner, group, and others in sets of three:

• The first set of three characters (e.g., rwx) represents the owner’s (user’s) permissions.

• The second set (e.g., r-x) represents the group’s permissions.

• The third set (e.g., r--) represents the permissions for others.

A - means there are no permissions for the corresponding operation. For instance, r-x means read and execute permissions are granted, but not write permissions.

You can view permissions on a Linux machine using the ls -l command:

$ ls -l
-rw-r--r--  1 user group  4096 Jun 29  01:49 foo.txt

In the example output above, -rw-r--r-- represents the permissions, user is the username of the owner, and group is the group name for the corresponding group.

Modifying Ownership and Permissions

You can change the ownership and group of a file using the following commands:

• chown user file: Changes the owner of a file.

• chgrp group file: Changes the group of a file.

• chown user:group file: Changes the owner and group of the file.

Typically, the chgrp command can be used without sudo if the user is changing to a group they belong to, while chown almost always requires sudo (superuser do) for changing the ownership of a file.

Permissions can be changed using the chmod command in two main formats:

• Symbolic Mode: Using letters to specify and modify permissions. For instance:

  • chmod u+w file: Adds write permissions for the user (owner) of the file.

  • chmod g-x file: Removes execute permissions from the group on the file.

  • chmod o=r file: Sets read-only permission for others on the file.

  • chmod a+rw file: Adds read and write permissions for all (user, group, others) on the file.

• Numerical (Octal) Mode: Uses numbers to specify permissions.

  • Each permission type is represented by a number: read (r) is 4, write (w) is 2, and execute (x) is 1.

  • Permissions are summed up for each entity. For example, rwx (read, write, and execute) is 4+2+1 = 7.

  • chmod 755 file: Sets permissions to rwxr-xr-x (7 for user, 5 for group, 5 for others).

Special Permissions

There are a few special permissions in Linux:

1. Setuid (set user ID): When set on an executable file, it allows the file to be executed with the permissions of the file’s owner. It is represented by an s in the owner’s execute position (e.g., rwsr-xr-x). It can be modified using chmod, as in chmod u+s file or by adding a 4 at the start of a permission set when using numerical mode with chmod, e.g., chmod 4755 file.

2. Setgid (set group ID): Similar to Setuid, it allows a file to be executed with the permissions of the group it belongs to (e.g., -rwxr-sr-x). It can be modified using chmod g+s file or by adding a 2 to the start of the permission set when using numerical mode, e.g., chmod 2755 file.

3. Sticky Bit: When set on a directory, it ensures that only the owner of the directory (or root) can delete or modify a file in the directory. It is represented as a t in the execute position of the others' permission set (e.g., drwxrwxrwt). It can be modified using chmod +t directory or by adding a 1 to the start of the permission set when using numerical mode, e.g., chmod 1755 directory. It only applies to directories.

Learn More

To learn more about Linux and Linux Permissions, you can explore these resources:

• RedHat: Linux Permissions Explained

• RedHat: Managing User and Groups in Linux

• ArchLinux: Users and groups

And you can always consult the Linux Manual Pages.

Cheers.