The Naked Tunnel — Build a mesh VPN

ping works. tcpdump on the wire shows your ICMP payload in the clear.

In the TUN lesson of the networking track you wrote a loop: read a packet from tun0, sendto it over UDP, recvfrom the peer's packets, write them to tun0. Run it on two machines and ping 100.64.0.2 gets replies. Then run tcpdump -X on the real interface, and there it is: inside each UDP datagram, a complete IP header, the ICMP header, and the eight bytes of pattern that ping stuffs into the payload. Anyone between the two machines reads it. Anyone who can reach your UDP port writes packets into your kernel.

Those two problems are what lessons 03 to 06 fix. Before fixing them, pin down exactly what the fix wraps. The repo's meshd has no plaintext mode, on purpose: there must never be a build of it that leaks. So this lesson's program is a 45-line naked.c that you write, linked against the repo's src/tun.c and src/util.c. The functions are the real ones. Only the glue is temporary.

tun.c: six functions, one of them a shell command

tun_open is the networking lesson's code with error logging. open("/dev/net/tun"), then ioctl(TUNSETIFF) with IFF_TUN | IFF_NO_PI: IP packets, not Ethernet frames, and no 4-byte packet-info prefix, so byte 0 of every read is the IP version nibble.

c

// src/tun.c
int tun_open(const char *name)
{
    int fd = open("/dev/net/tun", O_RDWR);
    if (fd < 0) {
        log_err("open /dev/net/tun: %m (need root or CAP_NET_ADMIN)");
        return -1;
    }
    struct ifreq ifr;
    memset(&ifr, 0, sizeof ifr);
    ifr.ifr_flags = IFF_TUN | IFF_NO_PI;
    snprintf(ifr.ifr_name, IFNAMSIZ, "%s", name);
    if (ioctl(fd, TUNSETIFF, &ifr) < 0) {
        log_err("TUNSETIFF %s: %m", name);
        close(fd);
        return -1;
    }
    return fd;
}

int tun_configure(const char *name, uint32_t overlay_ip, int prefix, int mtu)
{
    char ip[32], cmd[256];
    format_ip(overlay_ip, ip, sizeof ip);
    /* "replace" rather than "add" so a restart does not fail on EEXIST. */
    snprintf(cmd, sizeof cmd, "ip addr replace %s/%d dev %s", ip, prefix, name);
    if (system(cmd) != 0)
        return -1;
    snprintf(cmd, sizeof cmd, "ip link set %s mtu %d up", name, mtu);
    return system(cmd) == 0 ? 0 : -1;
}

bool ipv4_valid(const uint8_t *pkt, size_t len)
{
    if (len < 20 || (pkt[0] >> 4) != 4)
        return false;
    size_t ihl   = (pkt[0] & 0x0f) * 4u;
    size_t total = (size_t)pkt[2] << 8 | pkt[3];
    return ihl >= 20 && total >= ihl && total <= len;
}

/* IP headers are big-endian: the only place in this project we read a
 * number that way, which is why wire.h's LE helpers are not used here. */
static uint32_t be32(const uint8_t *p)
{
    return (uint32_t)p[0] << 24 | (uint32_t)p[1] << 16 |
           (uint32_t)p[2] << 8  | (uint32_t)p[3];
}

uint32_t ipv4_src(const uint8_t *pkt)   { return be32(pkt + 12); }
uint32_t ipv4_dst(const uint8_t *pkt)   { return be32(pkt + 16); }
uint8_t  ipv4_proto(const uint8_t *pkt) { return pkt[9]; }
          

tun_configure does what you typed by hand in the networking lesson, from C, through system(). Tailscale uses netlink for this; the shell commands are here because you can read them in a log. The prefix is always 10: that one ip addr installs the 100.64.0.0/10 dev tun0 route, and from then on the kernel delivers every overlay packet into your read.

The MTU is 1400, from TUN_MTU in wire.h. The inner packet you read will later be wrapped in a 13-byte header plus a 16-byte tag (lesson 05), then 8 bytes of UDP and 20 of outer IP, and sometimes a 33-byte relay header (lesson 10). That is 90 bytes of overhead on a 1500-byte link; 1400 leaves room for all of it and some slack. WireGuard uses 1420 because it has no relay layer.

ipv4_valid is the gatekeeper. Every byte that enters the daemon from either direction passes through it before any field is read. It checks four things: at least 20 bytes, version nibble 4, a header length that makes sense, and a total length that fits inside what was actually read. ipv4_src, ipv4_dst and ipv4_proto then read offsets 12, 16 and 9: the header you parsed byte by byte in networking lesson 01.

⚠️

The kernel will hand you packets that are not IPv4. The moment tun0 comes up, Linux sends IPv6 router solicitations and multicast listener reports through it. Their version nibble is 6. Without ipv4_valid you would read offset 16 of an IPv6 header and get half of a 128-bit address, then try to route it. The real daemon counts and drops these silently; so does naked.c.

Endpoints in host byte order

util.h defines struct endpoint { uint32_t ip; uint16_t port; }, and both fields are in host byte order. That is a deliberate break from the sockaddr_in habit. The daemon compares endpoints, prints them, stores them in the peer table and parses them from text constantly; doing all of that in network order means ntohs in every printf and a bug the first time you forget. Instead the conversion happens in exactly two functions, endpoint_to_sockaddr and endpoint_from_sockaddr, at the sendto and recvfrom boundary. udp_send(fd, &ep, buf, len) wraps the first; you will see it in every lesson from here on.

The loop

Here is naked.c, the whole file. It opens and configures the TUN, binds a UDP socket on the port you give it, and runs the two-fd poll loop from the networking track, with the repo's functions in the places where the networking version had raw syscalls.

c

// naked.c — this lesson only; not in the repo. Build against src/tun.o and src/util.o.
#include "tun.h"
#include "util.h"
#include "wire.h"          // TUN_MTU, MAX_PKT
#include <poll.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <unistd.h>

int main(int argc, char **argv)
{
    if (argc != 4)
        die("usage: naked OVERLAY_IP PORT PEER_IP:PORT");
    uint32_t overlay;
    struct endpoint peer;
    if (!parse_ip(argv[1], &overlay) || !parse_endpoint(argv[3], &peer))
        die("bad address");

    int tun = tun_open("tun0");
    if (tun < 0 || tun_configure("tun0", overlay, 10, TUN_MTU) < 0)
        return 1;

    int udp = socket(AF_INET, SOCK_DGRAM, 0);
    struct endpoint any = { .ip = 0, .port = (uint16_t)atoi(argv[2]) };
    struct sockaddr_in sa;
    endpoint_to_sockaddr(&any, &sa);
    if (bind(udp, (struct sockaddr *)&sa, sizeof sa) < 0)
        die("bind: %m");

    uint8_t pkt[MAX_PKT];
    struct pollfd fds[2] = {
        { .fd = tun, .events = POLLIN },
        { .fd = udp, .events = POLLIN },
    };
    for (;;) {
        if (poll(fds, 2, -1) < 0)
            continue;
        if (fds[0].revents & POLLIN) {                 // kernel → us → wire
            ssize_t len = read(tun, pkt, sizeof pkt);
            if (len > 0 && ipv4_valid(pkt, (size_t)len))
                udp_send(udp, &peer, pkt, (size_t)len);   // ← nothing happens here. yet.
        }
        if (fds[1].revents & POLLIN) {                 // wire → us → kernel
            ssize_t len = recv(udp, pkt, sizeof pkt, 0);
            if (len > 0 && ipv4_valid(pkt, (size_t)len))
                write(tun, pkt, (size_t)len);            // ← from anyone, unchecked. yet.
        }
    }
}
          

You saw this frame in the networking track. Step through it again, and this time read the daemon's two file descriptors as tun and udp in the code above: the 30-byte inner packet is what read(tun) returns, and udp_send is the moment it becomes somebody else's payload.

tun-path — one packet through the naked tunnel

Click next to start the walkthrough.

What meshd does differently, line by line

Now put the real daemon's handlers next to it. cmd_up in src/meshd.c polls four descriptors, not two: the TUN, the UDP socket, the TCP connection to coordd (lesson 07) and a unix socket for meshd status (lesson 12). The two that matter today are the same two fds you just saw, and their handlers differ from naked.c in exactly the places marked with arrows.

c

// src/meshd.c — the DATA branch of handle_packet() trimmed to its spine; see the repo
static void on_tun_readable(struct node *n, int64_t now)
{
    uint8_t pkt[MAX_PKT];
    ssize_t len = read(n->tun_fd, pkt, sizeof pkt);
    if (len <= 0 || !ipv4_valid(pkt, (size_t)len))
        return;                                /* e.g. IPv6 from the kernel */
    struct peer *p = lookup_by_overlay_ip(&n->peers, ipv4_dst(pkt));   // ← lesson 03
    if (!p) {
        n->drops.no_peer++;
        return;
    }
    path_send_data(n, p, pkt, (size_t)len, now);                      // ← lessons 05, 09
}

/* inside handle_packet(), for a packet whose first byte is PKT_DATA */
struct session *s;
struct peer *p = lookup_by_rx_index(&n->peers, get_u32le(pkt + 1), &s, now);  // ← lesson 06
uint8_t plain[MAX_PKT];
int plen = p ? session_decrypt(s, pkt, len, plain, now) : -1;      // ← lesson 05
if (plen < 0) { n->drops.bad_auth++; return; }
if (plen == 0) return;                                           /* keepalive */
if (!ipv4_valid(plain, (size_t)plen) || !peer_allows_src(p, ipv4_src(plain))) {
    n->drops.bad_src++;                                               // ← lesson 03
    return;
}
if (write(n->tun_fd, plain, (size_t)plen) < 0)
    log_warn("tun write: %m");
          

Read the shape, not the names. Outbound: read, validate, decide who this is for, encrypt and send to wherever that peer currently is. Inbound: decide who sent this, decrypt and check the tag, check they are allowed to send from that address, write. Four decisions, four lessons. The read and the write at either end are the same two syscalls as naked.c, and ipv4_valid guards both of them. The first byte of a received datagram is now a packet type, which is why there is a get_u32le(pkt + 1): the inner IP packet no longer starts at byte 0 of the datagram. From lesson 05 on, nothing on the wire does.

🚫

The naked tunnel is a remote packet injector. The write(tun, pkt) in naked.c accepts a packet from any address that sends to your UDP port and hands it to the kernel as if it arrived on tun0. A stranger can address a TCP SYN to 100.64.0.1:22, with any source they like, and your kernel will process it. Never run this on an interface reachable from the internet. The lab below uses two namespaces joined by a private cable precisely so that nothing else can reach the port.

💡

The data plane is a loop over two file descriptors. Every version of this program, from naked.c to the final meshd to wireguard-go, is poll on a TUN fd and a UDP fd, with a function between read and sendto. What the course adds is that function.

Run it

Two network namespaces, n1 and n2, joined by a veth pair: a virtual cable with 192.168.100.1 on one end and 192.168.100.2 on the other. Those are the "real" addresses. Each namespace runs one naked with an overlay address and the other end's real address as its peer. make first, so src/tun.o and src/util.o exist; util.o pulls in libsodium for its base64 helpers, hence -lsodium even though naked.c encrypts nothing. Everything below needs root.

sh

make
gcc -std=c11 -O2 -Wall -Wextra -D_GNU_SOURCE -Isrc -o naked naked.c src/tun.o src/util.o -lsodium

# the "internet": one cable between two namespaces
sudo ip netns add n1; sudo ip netns add n2
sudo ip link add v1 type veth peer name v2
sudo ip link set v1 netns n1; sudo ip link set v2 netns n2
sudo ip netns exec n1 ip addr add 192.168.100.1/24 dev v1
sudo ip netns exec n2 ip addr add 192.168.100.2/24 dev v2
sudo ip netns exec n1 ip link set v1 up; sudo ip netns exec n1 ip link set lo up
sudo ip netns exec n2 ip link set v2 up; sudo ip netns exec n2 ip link set lo up

# one tunnel end per namespace: OVERLAY_IP PORT PEER_IP:PORT
sudo ip netns exec n1 ./naked 100.64.0.1 51820 192.168.100.2:51820 &
sudo ip netns exec n2 ./naked 100.64.0.2 51820 192.168.100.1:51820 &

# watch the wire in n2 while n1 pings the overlay address
sudo ip netns exec n2 tcpdump -ni v2 -X -c 1 udp port 51820 &
sudo ip netns exec n1 ping -c 2 -p 4d4553484d455348 100.64.0.2      # payload "MESHMESH"
          

sh

PING 100.64.0.2 (100.64.0.2) 56(84) bytes of data.
64 bytes from 100.64.0.2: icmp_seq=1 ttl=64 time=0.21 ms
64 bytes from 100.64.0.2: icmp_seq=2 ttl=64 time=0.19 ms

IP 192.168.100.1.51820 > 192.168.100.2.51820: UDP, length 84
    0x0000:  4500 0070 3a7c 4000 4011 .... c0a8 6401   # outer IP: 112 bytes, proto 17, 192.168.100.1
    0x0010:  c0a8 6402 ca6c ca6c 005c .... 4500 0054   # → 192.168.100.2, port 51820 → 51820, then the INNER IP header
    0x0020:  5e11 4000 4001 .... 6440 0001 6440 0002   # inner proto 1 (ICMP), 100.64.0.1 → 100.64.0.2
    0x0030:  0800 .... 0c2a 0001 .... .... .... ....   # ICMP echo request
    ...
    0x0060:  4d45 5348 4d45 5348 4d45 5348 4d45 5348   MESHMESHMESHMESH
          

Byte 28 of the datagram, where UDP's payload begins, is 45: version 4, header length 5. Sixteen bytes later: 6440 0001, which is 100.64.0.1. Then 0800, an echo request, and the pattern you asked ping for, readable as ASCII in the right-hand column. In the networking lesson this was a diagram. Here it is your traffic on a cable, and anyone holding the cable has it. The repo's test/run.sh runs this same capture against the finished meshd and fails the build if MESHMESH appears on the wire even once. By lesson 05 it will not.

Clean up with sudo pkill naked; sudo ip netns del n1; sudo ip netns del n2. Deleting a namespace removes its interfaces, including tun0.

one-line takeaway

The tunnel is read(tun), udp_send, recv, write(tun) around a poll; everything the next ten lessons add goes into the gap between the read and the send.