Build a mesh VPN · lesson 01

The plan

You finished the networking track with a tunnel that pinged. This course turns it into three programs that connect machines behind two NATs, encrypted end to end, with nobody in the middle who can read a byte. First: what exactly are we building, and who are we hiding from.

in progress
15 min interactive

Your VPN works. The person who runs it reads your packets.

Picture the ordinary VPN: a server somewhere, and every device opens a tunnel to it. Your laptop sends a packet to your home server. The packet goes to the VPN company's box, which decrypts it, looks at the destination, encrypts it again and forwards it. For a moment, inside that box, your packet is plaintext. Whoever runs the box, or breaks into it, reads traffic between every pair of your devices. Every byte also makes a detour through a data center, so two machines on the same Wi-Fi talk to each other through another continent.

That design is a hub. The alternative is a mesh: every pair of nodes gets its own tunnel, encrypted with a key only those two nodes hold. There is still a server, but it does a different job. It introduces nodes to each other and then gets out of the way. It never sees a data packet. That is what Tailscale does, and it is what you build in this course, in C, with one library.

Control plane and data plane

Two kinds of traffic never mix in this system. The control plane is text about the network: who exists, which public key they have, which addresses they may use, where they were last seen. It goes over TCP to the coordination server. The data plane is the packets your applications send: IP packets read from tun0, encrypted, wrapped in UDP, sent straight to the other node. The coordination server is not on that path. If you unplug it, existing tunnels keep working until a key needs refreshing.

Three programs, one each for the three roles. You write all of them.

program runs on job Tailscale name
meshd every node opens tun0, finds peers, handshakes, encrypts, moves packets tailscaled
coordd one public host authenticates nodes, assigns overlay IPs, distributes the peer list, enforces the ACL control server
relayd one public host forwards already-encrypted packets when two NATs refuse a direct path DERP

Step through the visualization below. Dashed teal lines are the control plane, solid blue lines are the data plane. Watch where a data packet goes, and where it never goes.

mesh architecture — who talks to whom
Click next to start the walkthrough.
💡
The sentence the whole course hangs on. The public key is the node's identity. The coordination server only introduces. The data plane is peer to peer, encrypted end to end. The relay is a dumb pipe. Every design decision in the next eleven lessons follows from one of those four clauses.

Who sees what

A security design starts by naming the people you are hiding from and what each of them gets to see. Here is ours. Lessons 05, 06 and 10 each come back to one row of this table when they build the piece that enforces it.

who sees
your ISP / coffee-shop Wi-Fi outer IP:port pairs, packet sizes, timing; nothing inside
coordination server public keys, overlay IPs, endpoint candidates; never a data packet
relay [7][dst key][ciphertext]; cannot decrypt, cannot forge
a peer on the mesh only packets addressed to it, only from keys it was told about
someone who steals a private key can impersonate that node until it is removed from coordd

Read the last row twice. There is no password, no certificate authority, no login. A node is its 32-byte private key. Lose the file and someone else is you. That is also WireGuard's model, and the reason the key file is written with mode 0600 and the program refuses to overwrite one that exists.

Overlay addresses

Every node gets an address in 100.64.0.0/10: 100.64.0.1 for the first node to join, 100.64.0.2 for the second, and so on. The coordination server hands them out and remembers them, so a key keeps its address across restarts. Applications on the mesh only ever see these addresses. The real ones, the 192.168.1.23 behind one NAT and the 10.0.0.5 behind another, are the daemon's problem.

Why that block? 100.64.0.0/10 is "shared address space", reserved by RFC 6598 for carrier-grade NAT. It is not routable on the public internet, so a route for it on tun0 steals nothing from the real world. And unlike 10.0.0.0/8 or 192.168.0.0/16, it is almost never used on a home or office LAN, so the overlay does not collide with the network a node is actually plugged into. Tailscale picked the same block for the same two reasons. From the routing lesson: one route, 100.64.0.0/10 dev tun0, and the kernel delivers four million addresses into your read.

The repo

The reference implementation lives in the project repo. Every lesson from here on shows the real functions from it and ends with a command you can run. Here is the shape of the tree; each file is one lesson's territory.

sh
Makefile gcc -std=c11 -O2 -Wall -Wextra -D_GNU_SOURCE, -lsodium src/ util.h/.c log_*(), now_ms(), now_ns(), endpoint parsing, base64 tun.h/.c tun_open(), ipv4_src/dst/proto(), ipv4_valid() # lesson 02 wire.h packet types, sizes, little-endian helpers peers.h/.c struct peer, allowed-IPs, longest-prefix lookup # lesson 03 crypto.h/.c X25519 keypairs, base64 key text, key files (0600) # lesson 04 session.h/.c DATA encrypt/decrypt, tx counter, 64-bit replay window # lesson 05 handshake.h/.c INIT/RESP, crypto_box sealing, crypto_kx session keys # lesson 06 control.h/.c coordd TCP client: HELLO/ENDPOINTS/ROUTES/PING, PEER lines coordd.c coordination server (single file) # lesson 07 stun.h/.c STUN_REQ/STUN_RESP # lesson 08 path.h/.c candidates, hole punching, keepalives, roaming, relay # lessons 09–10 relayd.c relay (single file) # lesson 10 status.h/.c unix socket for `meshd status` # lesson 12 node.h struct node: the daemon's state, shared by the modules meshd.c subcommands, config, the poll() loop test/ netns-up.sh two NATed namespaces + a "cloud" bridge, on one Linux host run.sh end-to-end test (direct, relay, routes, ACL)

Three things about the build. One dependency: libsodium, for X25519, ChaCha20-Poly1305 and base64. You will not implement a cipher in this course; you will use the right ones correctly, which is the harder skill. One compiler invocation per file, with -Wall -Wextra and no suppressed warnings. And one Makefile rule that makes every object depend on every header, because a stale object in a course repo costs more than a one-second rebuild.

make
# Makefile CC = gcc CFLAGS = -std=c11 -O2 -Wall -Wextra -D_GNU_SOURCE LDLIBS = -lsodium COMMON = src/util.o src/crypto.o src/stun.o MESHD = src/meshd.o src/tun.o src/peers.o src/session.o src/handshake.o \ src/path.o src/control.o src/status.o $(COMMON) COORDD = src/coordd.o $(COMMON) RELAYD = src/relayd.o src/util.o all: meshd coordd relayd meshd: $(MESHD) $(CC) -o $@ $^ $(LDLIBS) # Every object depends on every header: cheap, and never stale. src/%.o: src/%.c $(wildcard src/*.h) $(CC) $(CFLAGS) -c -o $@ $< test: all ./test/run.sh

meshd.c, before the loop

meshd is one binary with four subcommands, dispatched in main by comparing argv[1]. Two of them never touch the network: genkey prints a fresh private key, or writes it to a file if you give a path; pubkey reads a private key on stdin and prints the matching public key. status asks a running daemon for its peer table over a unix socket. up is the daemon. The whole rest of the course is inside cmd_up.

c
// src/meshd.c static void usage(void) { fprintf(stderr, "usage: meshd genkey [path]\n" " meshd pubkey < private.key\n" " meshd up --key FILE --coord HOST:PORT --authkey SECRET\n" " [--relay HOST:PORT] [--port 51820] [--advertise CIDR,...]\n" " [--tun tun0]\n" " meshd status\n"); exit(2); } static int cmd_genkey(int argc, char **argv) { uint8_t pk[KEY_LEN], sk[KEY_LEN]; crypto_keypair_generate(pk, sk); if (argc > 2) return crypto_save_private(argv[2], sk) ? 0 : 1; // O_EXCL, mode 0600 char b64[KEY_B64_LEN]; crypto_key_to_b64(sk, b64); printf("%s\n", b64); return 0; } int main(int argc, char **argv) { if (argc < 2) usage(); crypto_init(); // sodium_init(), or die signal(SIGPIPE, SIG_IGN); // a dead control socket is not fatal if (!strcmp(argv[1], "genkey")) return cmd_genkey(argc, argv); if (!strcmp(argv[1], "pubkey")) return cmd_pubkey(); if (!strcmp(argv[1], "status")) return status_query(status_sock_path()); if (!strcmp(argv[1], "up")) return cmd_up(argc, argv); usage(); return 2; }

cmd_up parses its flags into one struct node, declared in src/node.h. That struct holds everything the daemon knows: both keys, the tun and UDP file descriptors, the coordination server's address, the peer table, drop counters. Every module gets a pointer to it instead of reaching for globals. You will read it field by field as the lessons add them; for now, know that --key, --coord and --authkey are required and everything else has a default: UDP port 51820 (WireGuard's), interface tun0, no relay. One convention to remember: coordd answers the public-endpoint probe on UDP at its TCP port plus one, so --coord host:7000 implies UDP 7001.

⚠️
meshd up needs root or CAP_NET_ADMIN. It opens /dev/net/tun and runs ip addr, ip link and ip route through system(). A product would speak netlink and drop privileges; the course keeps the shell commands because you can read them. genkey, pubkey and the build need no privileges at all.

The loop you will repeat twelve times

Every lesson ends the same way: make, then run something, then look at the result with tcpdump, meshd status or ip route. From lesson 07 on, "run something" means the lab in test/: two Linux network namespaces, each behind its own masquerading NAT that drops unsolicited inbound packets like a home router, joined by a bridge that plays the internet, with coordd and relayd on it. test/run.sh builds that lab, starts everything, pings across both NATs and checks with tcpdump that only encrypted packets crossed the bridge. It exits non-zero when anything fails, so it is also the course's test suite.

The lab needs root and Linux. On macOS, or on a laptop you do not want to hand root to, run it inside the privileged Docker container defined in test/Dockerfile: docker build -t mesh-lab test/ once, then docker run --rm --privileged -v "$PWD":/work -w /work mesh-lab sh -c 'make && ./test/run.sh'. Lesson 02 shows the namespace commands by hand, so you see what the script hides.

✅
What we deliberately do not build. WireGuard's Noise-IK handshake (ours is a two-message authenticated key exchange without identity hiding), netlink (we shell out to ip), IPv6, more than one relay, key expiry, SSO login. Lesson 12 lists each gap with what Tailscale does instead. Everything else, the parts that make packets cross two NATs encrypted, is real.

Run it

Clone, build, make an identity. The last command runs meshd with no arguments: it should print the usage text and exit with status 2. That is your smoke test that the binary links against libsodium and dispatches subcommands.

sh
sudo apt install gcc make libsodium-dev # Fedora: dnf install gcc make libsodium-devel git clone https://github.com/tdermendjiev/mesh-vpn cd mesh-vpn make mkdir -p ~/.mesh ./meshd genkey ~/.mesh/private.key # writes one base64 line, mode 0600 ./meshd pubkey < ~/.mesh/private.key # your identity: 44 chars ./meshd; echo "exit $?"
sh
gcc -std=c11 -O2 -Wall -Wextra -D_GNU_SOURCE -c -o src/meshd.o src/meshd.c ... gcc -o meshd src/meshd.o src/tun.o src/peers.o ... src/stun.o -lsodium gcc -o coordd src/coordd.o src/util.o src/crypto.o src/stun.o -lsodium gcc -o relayd src/relayd.o src/util.o -lsodium 6Y2kPq1x4QFJt0b9vWmZkR3cH7nLsA8dE2uG5iT1oXw= usage: meshd genkey [path] meshd pubkey < private.key meshd up --key FILE --coord HOST:PORT --authkey SECRET [--relay HOST:PORT] [--port 51820] [--advertise CIDR,...] [--tun tun0] meshd status exit 2

Your public key will differ, of course. Run genkey with the same path again and it fails with create ~/.mesh/private.key: File exists: the file is opened with O_EXCL so an identity is never silently replaced. Run ls -l ~/.mesh and check the mode is -rw-------. Do not run meshd up yet: it wants a coordination server, and you have not written one. Next lesson you take tun.c and a UDP socket and build the tunnel with nothing in it.

one-line takeaway

A mesh VPN is two planes that never meet: a server that only introduces nodes by public key, and encrypted UDP that goes straight from one node to the other.

next → The naked tunnel