Build a mesh VPN · lesson 06

The handshake

Lesson 05 encrypted every packet with tx_key and rx_key and said they come from the handshake. Why not derive them once from the two static keys and be done? Because then one stolen file decrypts every capture ever made. This lesson is handshake.c: 150 lines that make session keys disposable.

in progress
22 min interactive

Why not just use the static keys?

You could encrypt right now. Alice holds aSk and bPk, Bob holds bSk and aPk, and lesson 04 showed both compute the same 32 bytes. Hash them into a key, hand it to session_encrypt, no extra messages. Three things go wrong, and each is a field in the handshake.

No forward secrecy. The static secret is the same in every capture ever recorded. Steal private.key once and every DATA packet ever sent to that node opens. The fix is an ephemeral keypair per session, wiped after.

No freshness. A fixed key means every restart begins at counter 0 under the same key, the nonce reuse lesson 05 called fatal, and an attacker can replay an entire old conversation. The fix is a timestamp that must always increase.

No way to find the session. The receiver must know which keys to try before it can decrypt, and the source address is useless: NAT changes it, and lesson 09 lets peers roam. The fix is an index each side picks for the other to put in every packet.

Ephemeral key, index, timestamp: 32 + 4 + 8 = 44 bytes. The handshake exchanges them so that only the right peer can read them and only the right peer could have written them.

INIT and RESP: 117 bytes each

c
// src/wire.h #define KEY_LEN 32 /* X25519 key, both kinds */ #define NONCE_LEN 24 /* crypto_box nonce */ #define BOX_PLAIN_LEN (KEY_LEN + 4 + 8) /* eph_pub ‖ index ‖ ts */ #define BOX_LEN (BOX_PLAIN_LEN + 16) /* + crypto_box MAC = 60 */ /* [type 1][sender_static_pub 32][nonce 24][box 60] = 117 bytes. */ #define HANDSHAKE_LEN (1 + KEY_LEN + NONCE_LEN + BOX_LEN) // src/handshake.c #define OFF_STATIC 1 #define OFF_NONCE (OFF_STATIC + KEY_LEN) /* 33 */ #define OFF_BOX (OFF_NONCE + NONCE_LEN) /* 57 */

Type byte, 1 for HANDSHAKE_INIT or 2 for HANDSHAKE_RESP. The sender's static public key, 32 bytes, in the clear: this is how the receiver knows which peer-table entry to check. A 24-byte random nonce for crypto_box. Then the box: the 44 plaintext bytes plus a 16-byte authentication tag, 60 bytes. Total 117. INIT and RESP have the same layout; what differs is who sends them and what the timestamp means.

Step through the visualization. The byte grid uses the networking track's 16-per-row convention; purple is a public key, blue a header field the receiver reads, gray noise the sealed box, and green the moment a tag verifies.

handshake — INIT, RESP, session keys, rekey
Click next to start the walkthrough.

Sealing the box

crypto_box_easy is lesson 04's Diffie-Hellman at work: from the receiver's static public key and the sender's static private key it derives the static shared secret and encrypts-and-authenticates the plaintext under a 24-byte nonce. crypto_box_open_easy on the other side reaches the same secret, and the tag verifies only if the sealer held the sender's private key. One call, confidentiality and sender authentication. src/handshake.c wraps it in a build function shared by INIT and RESP.

c
// src/handshake.c void handshake_begin(struct peer *p) { memset(&p->hs, 0, sizeof p->hs); crypto_box_keypair(p->hs.eph_pk, p->hs.eph_sk); // fresh ephemeral keypair p->hs.local_index = randombytes_random(); p->hs.active = true; } /* Seals eph_pub ‖ index ‖ ts to the peer and writes the full packet. */ static size_t build(struct node *n, struct peer *p, uint8_t type, const uint8_t eph_pk[KEY_LEN], uint32_t index, uint64_t ts, uint8_t out[HANDSHAKE_LEN]) { uint8_t plain[BOX_PLAIN_LEN]; memcpy(plain, eph_pk, KEY_LEN); put_u32le(plain + KEY_LEN, index); put_u64le(plain + KEY_LEN + 4, ts); out[0] = type; memcpy(out + OFF_STATIC, n->pk, KEY_LEN); randombytes_buf(out + OFF_NONCE, NONCE_LEN); if (crypto_box_easy(out + OFF_BOX, plain, sizeof plain, out + OFF_NONCE, p->pubkey, n->sk) != 0) return 0; return HANDSHAKE_LEN; } size_t handshake_build_init(struct node *n, struct peer *p, uint8_t out[HANDSHAKE_LEN]) { uint64_t ts = now_ns(); if (ts <= p->hs.ts_last) /* clock did not move: force it */ ts = p->hs.ts_last + 1; if (p->hs.ts_first == 0) p->hs.ts_first = ts; p->hs.ts_last = ts; return build(n, p, PKT_HANDSHAKE_INIT, p->hs.eph_pk, p->hs.local_index, ts, out); }

struct handshake hs inside struct peer holds one attempt: the ephemeral keypair, the index, and the range of timestamps sent. Lesson 09 retransmits INIT every 500 ms, so key and index stay fixed across an attempt while each INIT gets a fresh now_ns(); otherwise the responder's replay guard would reject every retransmit. The box nonce is random, and may be: 24 bytes never collide in practice, and there is no counter to use yet.

Receiving INIT

c
// src/handshake.c static struct peer *open_box(struct node *n, const uint8_t *pkt, size_t len, uint8_t eph_pk[KEY_LEN], uint32_t *index, uint64_t *ts) { if (len != HANDSHAKE_LEN) return NULL; struct peer *p = lookup_by_pubkey(&n->peers, pkt + OFF_STATIC); if (!p) return NULL; /* unknown key: not our peer */ uint8_t plain[BOX_PLAIN_LEN]; if (crypto_box_open_easy(plain, pkt + OFF_BOX, BOX_LEN, pkt + OFF_NONCE, p->pubkey, n->sk) != 0) return NULL; /* forged or not for us */ memcpy(eph_pk, plain, KEY_LEN); *index = get_u32le(plain + KEY_LEN); *ts = get_u64le(plain + KEY_LEN + 4); return p; } int handshake_handle_init(struct node *n, const uint8_t *pkt, size_t len, struct peer **out, uint8_t resp[HANDSHAKE_LEN], int64_t now) { uint8_t their_eph[KEY_LEN]; uint32_t their_index; uint64_t ts; struct peer *p = open_box(n, pkt, len, their_eph, &their_index, &ts); if (!p) return -1; /* Replay guard: every INIT from this key must carry a newer timestamp. */ if (ts <= p->last_handshake_init_ts) return -1; p->last_handshake_init_ts = ts; *out = p; /* Both sides initiating at once: the smaller public key keeps going, * the larger one abandons its attempt and answers. */ if (p->hs.active && crypto_key_compare(n->pk, p->pubkey) < 0) return 0; p->hs.active = false; struct session s = { .initiator = false, .remote_index = their_index }; s.local_index = randombytes_random(); uint8_t eph_pk[KEY_LEN], eph_sk[KEY_LEN]; crypto_box_keypair(eph_pk, eph_sk); if (crypto_kx_server_session_keys(s.rx_key, s.tx_key, eph_pk, eph_sk, their_eph) != 0) return -1; /* degenerate ephemeral key */ install_session(p, &s, now); build(n, p, PKT_HANDSHAKE_RESP, eph_pk, s.local_index, ts, resp); sodium_memzero(eph_sk, sizeof eph_sk); return 1; }

The order of checks is the design. Length must be exactly 117. The static key at offset 1 must be in the peer table, so a stranger's INIT costs one lookup. The box must open, which proves the sender holds the matching private key. Only now are the 44 bytes inside trusted.

The timestamp guard. last_handshake_init_ts is the newest timestamp accepted from this key; an equal or older one is a replay, or a retransmit that already succeeded, and is dropped. It sits inside the box, so a recorded INIT cannot be bumped without the sender's static private key. The visualization's last step is this rejection.

The tie-break. Two nodes with a staged packet and no session both send INIT at once; if each responded there would be two sessions and no agreement on which to use. The rule: with an attempt in flight (hs.active) and the lexicographically smaller public key, I ignore your INIT and wait for your RESP. You, with the larger key, abandon your attempt and answer. Same decision on both sides, from the same two public keys, no extra message. Return value 0 tells meshd.c "genuine, but send nothing".

Session keys. The responder generates its own ephemeral keypair and calls crypto_kx_server_session_keys: X25519 between the two ephemeral keys, hashed with both public keys into two 32-byte outputs, one per direction. The "server" and "client" variants swap the directions, so the responder's rx_key is the initiator's tx_key. The static keys authenticated; the ephemeral keys alone make the session keys. Then sodium_memzero erases the ephemeral private key, and nothing on disk can recreate the session.

💡
Two Diffie-Hellmans with two jobs. Static × static, inside crypto_box, answers "is this really Alice?" Ephemeral × ephemeral, inside crypto_kx, answers "what key do we use for the next two minutes?" Mixing the two would tie the session keys to the long-term secret and lose forward secrecy. Keeping them separate is what makes the ephemeral private key safe to erase.

Receiving RESP and installing the session

c
// src/handshake.c /* cur becomes prev (kept for 10 s), the new session becomes cur. */ static void install_session(struct peer *p, const struct session *s, int64_t now) { p->prev = p->cur; p->prev.retire_ms = now + PREV_KEEP_MS; p->cur = *s; p->cur.established = true; p->cur.established_ms = now; p->hs.active = false; p->punching_since_ms = 0; /* lesson 09; ignore for now */ } struct peer *handshake_handle_resp(struct node *n, const uint8_t *pkt, size_t len, int64_t now) { uint8_t their_eph[KEY_LEN]; uint32_t their_index; uint64_t ts; struct peer *p = open_box(n, pkt, len, their_eph, &their_index, &ts); if (!p || !p->hs.active) return NULL; /* The echoed timestamp must be one we sent in THIS attempt. */ if (ts < p->hs.ts_first || ts > p->hs.ts_last) return NULL; struct session s = { .initiator = true, .remote_index = their_index, .local_index = p->hs.local_index }; if (crypto_kx_client_session_keys(s.rx_key, s.tx_key, p->hs.eph_pk, p->hs.eph_sk, their_eph) != 0) return NULL; install_session(p, &s, now); sodium_memzero(p->hs.eph_sk, sizeof p->hs.eph_sk); return p; } bool handshake_rekey_due(const struct peer *p, int64_t now) { return p->cur.established && p->cur.initiator && now - p->cur.established_ms >= REKEY_AFTER_MS; }

The responder echoes the initiator's timestamp, and the initiator checks it falls inside [ts_first, ts_last], the range sent in this attempt; a replayed RESP to last week's INIT fails and installs nothing. Then the client variant of crypto_kx with the ephemeral key held since handshake_begin gives the mirror image of the responder's keys, and the ephemeral private key is erased.

Indices. Each side chose a random 32-bit local_index and shipped it in its box; the other side stores it as remote_index and writes it into every DATA header. lookup_by_rx_index in peers.c matches it against cur and prev. Addresses never enter into it.

Timers. REKEY_AFTER_MS = 120 000: two minutes after install, the initiator, and only the initiator, sends a fresh INIT. REJECT_AFTER_MS = 180 000: session_usable refuses any session older than three minutes in both directions, so a failing rekey goes silent instead of running an old key forever; the minute between is the grace period. install_session keeps the old session as prev for PREV_KEEP_MS = 10 s, so DATA already in flight under the old index still decrypts.

⚠️
This responder switches to the new session the moment it sends RESP. If a DATA packet under the new keys overtakes the RESP on the wire, the initiator does not know the new index yet and drops it: one lost packet per rekey, at most. WireGuard avoids this by having the responder keep sending under the old session until the initiator's first DATA packet confirms the new one. That needs a third state in the session machine; the repo trades one packet every two minutes for not having it. Know the difference when you read the real implementation.
🚫
One key, one counter, one direction. A new session starts tx_counter at 0, which is safe only because the keys are new. Reuse the static secret as a session key and every restart repeats nonce 0 under the same key; send with prev after a rekey and two counters run under one key. In this repo path.c encrypts only with p->cur, and cur is a zeroed struct session at every install.
🚫
Secrets are compared in constant time or not at all. crypto_box_open_easy and the AEAD decrypt check tags with sodium_memcmp, which takes the same time wherever the mismatch is; a plain memcmp returns at the first wrong byte and leaks, through timing, how many bytes an attacker got right. This repo never touches a tag itself. It does memcmp in crypto_key_compare, and that is fine: those are public keys, already on the wire in the clear.

What Noise-IK adds

WireGuard's handshake is the IK pattern from the Noise framework: also two fixed-size messages, static authentication plus ephemeral forward secrecy, a timestamp replay guard, receiver indices, rekey at two minutes, reject at three. This design mirrors those choices on purpose. What it leaves out:

Identity hiding. Our static public key sits at offset 1 in the clear, so anyone on the path learns which key is talking; Noise-IK encrypts it so only the responder does. Key-compromise impersonation resistance. crypto_box is symmetric between two parties, so whoever steals Bob's private key can seal boxes to Bob that claim to be from anyone Bob knows; Noise-IK mixes the ephemeral keys into authentication so a stolen key lets you be Bob, not everyone Bob trusts. Cookie DoS protection. Opening a box costs a scalar multiplication, and a flood of INITs from a known key makes the responder pay it every time; WireGuard makes an overloaded responder demand a cheap cookie first.

Real gaps, all three. None changes the shape of what you built: two messages that turn two long-term identities into one short-lived pair of keys.

Run it

Start the lab and watch the handshake in both nodes' logs. The lab runs for a few minutes, so you can also see a rekey if you keep meshd status open.

sh
make sudo ./test/run.sh # leave it running; in a second terminal: grep "session with" /tmp/mesh-lab/meshd-A.log /tmp/mesh-lab/meshd-B.log MESHD_SOCK=/tmp/mesh-lab/A.sock ./meshd status ip netns exec cloud tcpdump -nn -i br0 'udp[8] = 1 or udp[8] = 2' # wait for the lab to restart B
sh
/tmp/mesh-lab/meshd-A.log:10:21:07.904 info session with 100.64.0.2 established (initiator, direct) /tmp/mesh-lab/meshd-B.log:10:21:07.903 info session with 100.64.0.1 established (responder, direct) node 100.64.0.1 pub kQ3vZl1y…CzE= udp 51820 public 10.0.0.2:51820 relay 10.0.0.1:7002 control connected peers 1 peer 100.64.0.2 7pL2mXaQ…FgI= path direct endpoint 10.0.0.3:51820 session 14s tx 6 rx 6 last_rx 2s allowed 100.64.0.2/32 candidates 10.0.2.2:51820,10.0.0.3:51820 drops no_peer 0 bad_src 0 bad_auth 0 no_session 1 10:21:52.118 IP 10.0.0.2.51820 > 10.0.0.3.51820: UDP, length 117 10:21:52.119 IP 10.0.0.3.51820 > 10.0.0.2.51820: UDP, length 117

B's line is a millisecond earlier than A's: B installed the session when it sent RESP, A when it received it. session 14s is now − established_ms; watch it reset once it passes 120 and A, the initiator, rekeys. no_session 1 is the first ping, staged until RESP arrived. Keys are shortened here; status prints all 44 characters. The live tcpdump stays quiet until the lab restarts B, then shows one INIT and one RESP, 117 bytes each. A nonzero bad_auth on an idle lab means someone sent you a box you could not open.

one-line takeaway

Static keys prove who is talking, inside a crypto_box; ephemeral keys, exchanged in that box and erased afterwards, make the session keys, so a stolen identity cannot open yesterday's traffic.