Finding Your Public Endpoint — Build a mesh VPN

The address you report is not the address you have

Run the two nodes from last lesson on two different home networks and look at meshd status on A. The peer line for B says candidates 10.0.2.2:51820. That is B's eth0 address. From A's network, 10.0.2.2 is either nothing at all or some unrelated printer. A will send its first packet there, and it will be dropped or misdelivered without leaving the building.

B did not lie. It called getifaddrs, got 10.0.2.2, and sent what it had. The NAT lesson explained the gap: the public ip:port that B's packets carry on the internet is written by B's router, into a translation table that B cannot read. No syscall on B returns it. getsockname returns the inside half of the row. The outside half is somebody else's memory.

But every packet B sends to the internet has that outside half stamped on it as the source. Anyone who receives one can read it off and send it back. In the NAT traversal lesson you wrote a rendezvous server that did exactly this with six raw bytes. coordd is already a machine every node can reach. So it grows one UDP socket, and the exchange gets a packet type.

STUN in eight bytes

The node sends one byte, [5], to coordd's UDP port. The server's recvfrom fills a sockaddr_in with the source address as the packet arrived, after every NAT on the path has rewritten it. The server copies that address and port into seven bytes, [6][ipv4 4][port 2], and sends them back. That is the whole of stun.c.

c

// src/stun.c
void stun_send_request(int udp_fd, const struct endpoint *server)
{
    uint8_t req[STUN_REQ_LEN] = { PKT_STUN_REQ };          // one byte: 5
    udp_send(udp_fd, server, req, sizeof req);
}

// [6][ipv4 4][port u16 BE]. The address is copied straight out of the
// sockaddr, which is why this one field is big-endian like IP itself.
size_t stun_build_response(const struct endpoint *from, uint8_t out[STUN_RESP_LEN])
{
    out[0] = PKT_STUN_RESP;
    out[1] = from->ip >> 24; out[2] = from->ip >> 16;
    out[3] = from->ip >> 8;  out[4] = from->ip;
    out[5] = from->port >> 8; out[6] = from->port;
    return STUN_RESP_LEN;
}

bool stun_parse_response(const uint8_t *pkt, size_t len, struct endpoint *ep)
{
    if (len != STUN_RESP_LEN || pkt[0] != PKT_STUN_RESP)
        return false;
    ep->ip = (uint32_t)pkt[1] << 24 | (uint32_t)pkt[2] << 16 |
             (uint32_t)pkt[3] << 8 | pkt[4];
    ep->port = (uint16_t)(pkt[5] << 8 | pkt[6]);
    return true;
}

// src/coordd.c — the server side, one case in the poll loop
static void on_udp_readable(int udp_fd)
{
    uint8_t pkt[64], resp[STUN_RESP_LEN];
    struct sockaddr_in sa;
    socklen_t salen = sizeof sa;
    ssize_t len = recvfrom(udp_fd, pkt, sizeof pkt, 0, (struct sockaddr *)&sa, &salen);
    if (len != STUN_REQ_LEN || pkt[0] != PKT_STUN_REQ)
        return;
    struct endpoint from;
    endpoint_from_sockaddr(&sa, &from);     // the NAT-rewritten address
    stun_build_response(&from, resp);
    udp_send(udp_fd, &from, resp, sizeof resp);
}
          

That is all of src/stun.c. Two details are worth a second look. The reply is big-endian, while every other multi-byte field in wire.h is little-endian. That is deliberate: these seven bytes are an IP address and a port, the same things an IP header carries in network order, and writing them any other way would invite a htons bug the moment someone copies them into a sockaddr. And the request is sent from n->udp_fd, the same socket that will carry handshakes and data. The NAT row is keyed by inside ip:port; a second socket would get a second row and an answer about a port nobody will ever use.

You saw this table in the NAT lesson with a web server on the far end. Step through it again, and this time read the row's outside half, 198.51.100.4:40001, as the seven bytes coordd sends back to the laptop.

nat — the row coordd reads back to you

Click next to start the walkthrough.

💡

Mental model: the server is a mirror. It knows nothing about the node that the node's own router did not write on the wire. It does not store the answer, log it, or correlate it; it reflects seven bytes and forgets. Real STUN (RFC 5389) is this exchange with a 20-byte header, a transaction ID and an XOR-obfuscated address attribute. Tailscale speaks real STUN to its DERP servers. Ours drops the ceremony and keeps the mechanism.

Candidates, plural

So the node now knows its public endpoint. Why not report only that? Because two laptops on the same office network would then try to reach each other through the office router's public address. Many consumer routers do not forward a packet that comes from inside and is addressed to their own outside: there is no row for it, and the packet is dropped. Even when it works, it is a trip to the router and back for two machines on the same switch. The LAN address is the right one for a LAN peer, the public address is the right one for everyone else, and the node does not know which kind of peer it is talking to. So it reports both and lets the peer try them all; how the peer picks is lesson 09.

c

// src/control.c
// Every non-loopback IPv4 address except the tunnel's own, each with our
// UDP port, then the STUN-discovered public endpoint.
static void endpoints_csv(struct node *n, char *out, size_t len)
{
    out[0] = '\0';
    struct ifaddrs *ifa, *list;
    if (getifaddrs(&list) != 0)
        return;
    for (ifa = list; ifa; ifa = ifa->ifa_next) {
        if (!ifa->ifa_addr || ifa->ifa_addr->sa_family != AF_INET ||
            strcmp(ifa->ifa_name, "lo") == 0 ||
            strcmp(ifa->ifa_name, n->tun_name) == 0)      // never offer 100.64.x.y
            continue;
        struct endpoint ep;
        endpoint_from_sockaddr((struct sockaddr_in *)ifa->ifa_addr, &ep);
        ep.port = n->port;                 // ifaddrs carry no port
        char text[32];
        format_endpoint(&ep, text, sizeof text);
        snprintf(out + strlen(out), len - strlen(out), "%s%s", out[0] ? "," : "", text);
    }
    freeifaddrs(list);
    if (n->have_public) {
        char text[32];
        format_endpoint(&n->public_ep, text, sizeof text);
        snprintf(out + strlen(out), len - strlen(out), "%s%s", out[0] ? "," : "", text);
    }
}

// src/meshd.c
static void on_stun_response(struct node *n, const uint8_t *pkt, size_t len)
{
    struct endpoint ep;
    if (!stun_parse_response(pkt, len, &ep))
        return;
    if (n->have_public && endpoint_equal(&ep, &n->public_ep))
        return;                              // same answer as last time: silence
    char text[32];
    log_info("stun: public endpoint %s", format_endpoint(&ep, text, sizeof text));
    n->public_ep = ep;
    n->have_public = true;
    control_send_endpoints(n);               // changed → tell the server
}
          

getifaddrs walks the kernel's interface list and hands back every address on every interface. The loop keeps IPv4, drops lo, and drops tun0 itself: offering 100.64.0.1:51820 as a place to reach the node that owns 100.64.0.1 would be a loop. Each address gets the node's own UDP port, because an interface address has no port and the socket is bound to one port on all of them. The public endpoint goes last. The result is the CSV in the ENDPOINTS line, and coordd forwards it verbatim in the third field of every PEER line about this node.

Refresh, and re-send only on change

Before the control connection, stun_discover_blocking sends a request and waits up to a second for the answer, three times. If none comes, that is a warning, not an error: a node without a public endpoint can still work on a LAN. Then in the poll() loop, tick() sends a fresh request every STUN_REFRESH_MS, 60 s. The reply usually says what the last one said, and on_stun_response drops it on the floor. When it differs, because the laptop moved to another network or the router gave up the old mapping and allocated a new port, the node logs it, stores it, and sends a new ENDPOINTS line. coordd re-broadcasts, and within one round trip every peer's candidates[] for this node is current. That one-minute loop is how the mesh follows a laptop around.

⚠️

Common mistake: trusting the port. The reply says the public port is 51820, the same as the local one, because most routers preserve the port when nothing else is using it. Do not hard-code that. Behind a carrier-grade NAT, or when two machines on the LAN picked the same port, the outside port is whatever the router had free, and 60 seconds later it can be different again. Always use the number in the reply.

✅

Rule: one socket for everything. STUN request, handshake, data, keepalive, relay: all through n->udp_fd. The public endpoint the server reported describes that socket's NAT row and no other. Every packet you send through it also refreshes that row's idle timer; a Linux router keeps a replied UDP flow for 180 s by default, so the 60 s STUN refresh usually gets the same public port back each time.

Run it

With coordd from last lesson still running, ask it the question by hand, then watch meshd ask it. The first command needs no root. The capture shows the exchange as the bytes it is: a 1-byte datagram out, a 7-byte datagram back.

sh

# by hand: one byte in, seven bytes out (coordd's UDP port is the TCP port + 1)
printf '\005' | nc -u -w 1 192.0.2.10 7001 | xxd

# watch meshd do it, from a third terminal, then start the node
sudo tcpdump -ni any -X udp port 7001
sudo ./meshd up --key ~/.mesh/private.key --coord 192.0.2.10:7000 --authkey lab-secret
sudo ./meshd status | head -1
          

sh

# xxd: type 6, then 198.51.100.4, then port 40001 (0x9c41), big-endian
00000000: 06c6 3364 049c 41                        ..3d..A

# tcpdump: the request and the reply, 1 and 7 bytes of UDP payload
12:41:07.210 IP 192.168.1.7.51820 > 192.0.2.10.7001: UDP, length 1
        0x0020:  .... 05
12:41:07.231 IP 192.0.2.10.7001 > 192.168.1.7.51820: UDP, length 7
        0x0020:  .... 06c6 3364 049c 41

# meshd's log, then status: the public endpoint is now a fact the node holds
12:41:07.232 info stun: public endpoint 198.51.100.4:40001
node 100.64.0.1 pub mK3f9Qx2…Yk= udp 51820 public 198.51.100.4:40001 control connected
          

On the coordination server's side, coordd's log shows nothing: it answered a UDP packet and moved on. Now look at meshd status on the other node. Its peer line for this one reads candidates 192.168.1.7:51820,198.51.100.4:40001: the LAN address it would need if it were in the same building, and the public one it needs from anywhere else. It has somewhere to send a packet. Whether a packet sent there arrives is the next lesson.

one-line takeaway

Your router writes your public address on every packet and you cannot read it; coordd reads it back in seven bytes, and you report it alongside every local address so LAN peers and distant peers each have a candidate that works.