Build a TCP/IP Stack from Scratch · Module 02

Hands-on Test: End-to-End ARP

Hands-on test: drive ARP end-to-end (client → your stack)

By default, packets from the client reach the stack container's eth0 (Docker bridge), not your tap0. To let the client's ARP request hit your TAP (and your code), we'll wire a temporary Layer-2 relay inside the stack container.

Heads-up: we'll create a Linux bridge br0 and attach eth0 and tap0 to it. This is safe in our lab, but it briefly reconfigures networking inside the stack container. If something goes wrong, a docker compose down && up -d restores the previous state.

8.1 Create a bridge and plug ports

Run inside the stack container:

docker compose exec stack bash -lc '
 set -e
 
 # 1) Create bridge
 ip link add name br0 type bridge
 ip link set br0 up
 
 # 2) Put eth0 into the bridge (move its IP off eth0 first)
 ETH_IP=$(ip -4 -o addr show dev eth0 | awk "{print \$4}" || true)
 if [ -n "$ETH_IP" ]; then ip addr del $ETH_IP dev eth0 || true; fi
 ip link set eth0 promisc on
 ip link set eth0 master br0
 
 # 3) Put tap0 into the bridge
 ip link set tap0 promisc on
 ip link set tap0 master br0
 
 # 4) Give the container an IP on the bridge so we can still reach it (optional)
 # We reuse the old eth0 IP, or assign 10.10.0.4/24 if empty.
 if [ -n "$ETH_IP" ]; then
 ip addr add $ETH_IP dev br0
 else
 ip addr add 10.10.0.4/24 dev br0 || true
 fi
 
 # 5) Keep your TAP host IP (for ARP target = 10.10.0.1)
 ip addr add 10.10.0.1/24 dev tap0 2>/dev/null || true
 ip link set br0 up
 ip -brief addr
'

What this does:

  • br0 acts like an internal switch.
  • Frames arriving on eth0 (from the client via Docker bridge) are forwarded to tap0, where your program is listening.
  • Your program can then craft an ARP reply back through tap0br0eth0 → client.

8.2 Run your stack and trigger ARP from the client

Start your program (keep it running):

docker compose exec stack bash -lc 'cd /stack && make && ./bin/stack'

In another terminal, send ARP from the client:

docker compose exec client arping -I eth0 -c 3 10.10.0.1

Expected in the stack logs:

ARP Request: who has 10.10.0.1? tell 10.10.0.3
Sent ARP reply (42 bytes)

Expected on the client:

ARPING 10.10.0.1 from 10.10.0.3 eth0
Unicast reply from 10.10.0.1 [..:..:..:..:..:..:..] 0.x ms
...

8.3 Verify on the wire (optional)

Capture on the stack side:

docker compose exec stack tcpdump -ni br0 -vvv arp

You should see a broadcast request (dst MAC ff:ff:ff:ff:ff:ff) and your unicast reply (dst = client MAC).

8.4 Rollback / cleanup (if needed)

If you want to undo the bridge wiring:

docker compose exec stack bash -lc '
 set -e
 ip link set eth0 nomaster 2>/dev/null || true
 ip link set tap0 nomaster 2>/dev/null || true
 ip addr flush dev br0 || true
 ip link del br0 2>/dev/null || true
 ip link set eth0 promisc off 2>/dev/null || true
 ip link set tap0 promisc off 2>/dev/null || true
'
# or simply restart the lab:
# cd lab && docker compose down && docker compose up -d

Automation Script

Now every time we make changes and rebuild the project we have a lot to do in order for our stack to run so we can automate things a bit.

Let's automate the whole wiring with a single runner script that:

  • builds the code,
  • starts ./bin/stack in the background (so it can create tap0),
  • waits for tap0,
  • brings tap0 up and assigns 10.10.0.1/24,
  • creates/updates a bridge br0,
  • moves eth0 and tap0 into br0,
  • restores the container IP on br0,
  • then attaches to the stack process and keeps its logs in your terminal.

1) Add a runner script

Create stack/run.sh:

#!/usr/bin/env bash
set -euo pipefail
 
# ---- config ----
STACK_BIN="/stack/bin/stack"
TAP_IF="tap0"
TAP_IP_CIDR="10.10.0.1/24"
BR="br0"
ETH="eth0"
# ----------------
 
echo "[run] building stack…"
make -C /stack -s
 
echo "[run] launching $STACK_BIN…"
$STACK_BIN &
STACK_PID=$!
 
# Wait for the program to create tap0 via /dev/net/tun
echo "[run] waiting for ${TAP_IF} to appear…"
for i in $(seq 1 50); do
 if ip link show "${TAP_IF}" &>/dev/null; then
 break
 fi
 sleep 0.1
done
ip link show "${TAP_IF}" >/dev/null || { echo "[run] ${TAP_IF} not found"; kill $STACK_PID || true; exit 1; }
 
# Put tap0 up and give it an IP (idempotent)
echo "[run] configuring ${TAP_IF}…"
ip link set "${TAP_IF}" up || true
ip addr add "${TAP_IP_CIDR}" dev "${TAP_IF}" 2>/dev/null || true
 
# Create bridge if missing
if ! ip link show "${BR}" &>/dev/null; then
 echo "[run] creating bridge ${BR}…"
 ip link add name "${BR}" type bridge
fi
ip link set "${BR}" up || true
 
# Remember eth0's IPv4 (if any), then move eth0 into bridge
ETH_IP=$(ip -4 -o addr show dev "${ETH}" | awk '{print $4}' || true)
if [ -n "${ETH_IP:-}" ]; then
 ip addr del "${ETH_IP}" dev "${ETH}" || true
fi
 
# Add eth0 to bridge (idempotent)
echo "[run] enslaving ${ETH} -> ${BR}…"
ip link set "${ETH}" promisc on || true
ip link set "${ETH}" master "${BR}" 2>/dev/null || true
 
# Add tap0 to bridge (idempotent)
echo "[run] enslaving ${TAP_IF} -> ${BR}…"
ip link set "${TAP_IF}" promisc on || true
ip link set "${TAP_IF}" master "${BR}" 2>/dev/null || true
 
# Put the container's IP on br0 so it's reachable
if [ -n "${ETH_IP:-}" ]; then
 ip addr add "${ETH_IP}" dev "${BR}" 2>/dev/null || true
else
 # fallback to compose-assigned address if needed
 ip addr add 10.10.0.4/24 dev "${BR}" 2>/dev/null || true
fi
 
echo "[run] final state:"
ip -brief addr show "${BR}" || true
ip -brief addr show "${TAP_IF}" || true
bridge link show 2>/dev/null | grep -E "(${ETH}|${TAP_IF})" || true
echo "[run] ready. forwarding client frames to ${TAP_IF}."
 
# Keep the stack process in the foreground (show its logs)
wait "${STACK_PID}"

Make it executable:

docker compose exec stack bash -lc 'chmod +x /stack/run.sh'

Now we can run the runner script and test from the client an ARPING command again and everything should work the same, but the process is more simple for us now to make changes and test.

Notes

  • The script is idempotent: re-running it keeps/reuses br0, re-enslaves interfaces, and reassigns IPs if needed.
  • To "reset" the wiring, you can docker compose down && docker compose up -d, or add a small stack/cleanup.sh later if you want finer-grained teardown.
  • We won't cover the explanation of the script but if you want to learn you can always look for bash scripting tutorials online and try to understand or even write the scripts yourself.

Success! Your stack now responds to ARP requests from other containers end-to-end!