poll() and Event Loops — Networking

Blocked on the wrong socket

The HTTP server from the last lesson has one loop: accept, then recv until the request is complete, then respond. Picture two clients. Client A connects and sends nothing. Your server is now inside recv(A), asleep, waiting for bytes that are not coming. Client B connects, sends a perfect request, and waits. Nobody is calling accept. Nobody is calling recv(B). B's bytes sit in a kernel receive buffer that your program never looks at.

The problem is not speed. The problem is that a blocking call on one fd makes the whole thread deaf to every other fd. You saw one fix in the systems track: a thread per client, each free to block on its own socket. That works, and costs a stack per client plus the locking you learned to fear. There is a second fix that costs almost nothing: never block on a socket. Block on the question instead.

Ask the kernel which fds are ready

The kernel already knows which receive buffers hold data and which send buffers have room. poll lets you hand it a list of fds and sleep until at least one of them is ready. One call, one thread, any number of sockets.

c

#include <poll.h>

struct pollfd {
    int   fd;        // the descriptor to watch
    short events;    // what you care about: POLLIN, POLLOUT
    short revents;   // what happened — filled in by the kernel
};

int poll(struct pollfd *fds, nfds_t nfds, int timeout_ms);
// returns: number of fds with nonzero revents, 0 on timeout, -1 on error
          

You fill in fd and events. The kernel fills in revents. POLLIN means a recv would return something right now — data, or a 0 for EOF. POLLOUT means a send would accept bytes without blocking. POLLHUP and POLLERR show up in revents whether you asked or not: the peer hung up, or the socket is broken. A listening socket reports POLLIN when a connection is waiting in the backlog, because accept would not block.

💡

poll() is a blocking call that blocks on all of them at once. Instead of sleeping inside recv(A), you sleep inside poll, which wakes the moment any watched fd changes state. The thread still sleeps most of the time — it just never sleeps on the wrong socket.

The loop

Step through the visualization. The table is your pollfd array. Each iteration: call poll, walk the array, act on every entry whose revents is nonzero, go back to poll. New connections add a row. Hang-ups remove one. The fourth row is a file descriptor that is not a socket at all — hold that thought for lesson 12.

poll() — one thread, many fds

Click next to start the walkthrough.

Readiness is a hint, not a promise

POLLIN told you a recv would not block. Usually true. Not always: between poll returning and your recv, the kernel can discard the data (a bad checksum caught late, a reset), or another thread can read it. If your socket is blocking, that rare case freezes the whole loop — exactly the bug you were fixing.

So an event loop puts every socket in non-blocking mode. A non-blocking recv with nothing to read does not sleep; it returns -1 with errno == EAGAIN (also spelled EWOULDBLOCK). You treat that as "nothing to do here" and move on. The same applies to accept on a non-blocking listener and send on a socket whose send buffer is full.

c

#include <fcntl.h>

static int set_nonblocking(int fd) {
    int flags = fcntl(fd, F_GETFL, 0);
    if (flags == -1) return -1;
    return fcntl(fd, F_SETFL, flags | O_NONBLOCK);
}
          

⚠️

EAGAIN is not an error. Treat it as "try later". New event-loop code often does if (n < 0) close(fd). On a non-blocking socket that drops healthy connections the first time readiness lies. Check errno: EAGAIN means go back to poll; anything else is a real error.

A complete echo server in one thread

This server handles as many clients as the pollfd array holds, with no threads and no client able to stall another. Index 0 is always the listening socket. When it reports POLLIN, accept a client, make it non-blocking, append it. For every other ready entry, recv and echo back. On EOF or hang-up, close and remove the row by swapping in the last one.

c

#include <stdio.h>
#include <errno.h>
#include <fcntl.h>
#include <poll.h>
#include <unistd.h>
#include <arpa/inet.h>
#include <sys/socket.h>

#define MAXFDS 64

// set_nonblocking() from above; send_all() from lesson 06

int main(void) {
    int ls = socket(AF_INET, SOCK_STREAM, 0);
    int yes = 1;
    setsockopt(ls, SOL_SOCKET, SO_REUSEADDR, &yes, sizeof yes);
    struct sockaddr_in a = { .sin_family = AF_INET, .sin_port = htons(7),
                             .sin_addr.s_addr = htonl(INADDR_ANY) };
    if (bind(ls, (struct sockaddr *)&a, sizeof a) == -1) { perror("bind"); return 1; }
    listen(ls, 16);
    set_nonblocking(ls);

    struct pollfd fds[MAXFDS];
    int nfds = 1;
    fds[0].fd = ls;
    fds[0].events = POLLIN;

    for (;;) {
        int ready = poll(fds, nfds, -1);       // -1: sleep until something happens
        if (ready == -1) { if (errno == EINTR) continue; perror("poll"); return 1; }

        // 1. new connection?
        if (fds[0].revents & POLLIN) {
            int c = accept(ls, NULL, NULL);
            if (c >= 0 && nfds < MAXFDS) {
                set_nonblocking(c);
                fds[nfds].fd = c;
                fds[nfds].events = POLLIN;
                fds[nfds].revents = 0;
                nfds++;
            } else if (c >= 0) {
                close(c);                              // table full: refuse
            }
        }

        // 2. walk the clients (backwards, so removal is safe)
        for (int i = nfds - 1; i >= 1; i--) {
            if (fds[i].revents == 0) continue;

            int drop = 0;
            if (fds[i].revents & (POLLHUP | POLLERR)) {
                drop = 1;
            } else if (fds[i].revents & POLLIN) {
                char buf[4096];
                ssize_t n = recv(fds[i].fd, buf, sizeof buf, 0);
                if (n > 0)        send_all(fds[i].fd, buf, n);
                else if (n == 0) drop = 1;                      // EOF
                else if (errno != EAGAIN && errno != EWOULDBLOCK) drop = 1;
                // EAGAIN: readiness lied — nothing to do, poll again
            }

            if (drop) {
                close(fds[i].fd);
                fds[i] = fds[nfds - 1];                      // swap last row in
                nfds--;
            }
        }
    }
}
          

Connect three terminals with nc localhost 7 and type in any of them. Each line comes back on the terminal that sent it. Leave one idle for an hour; the others are unaffected. Kill one; its row disappears and the loop continues. That is the behaviour the sequential server could not give you, and it still has one thread and one stack.

✅

Rule: in an event loop, nothing blocks except poll. Every socket is O_NONBLOCK. Every recv, send, and accept is prepared to get EAGAIN and give up until the next iteration. The moment one call can sleep, every other client waits for it.

Why a VPN daemon is an event loop

The echo server watches sockets. But poll does not care what kind of fd it is given — a pipe, a terminal, a device file all work. A mesh VPN daemon has exactly two fds that matter: a UDP socket carrying encrypted packets to and from other peers, and a TUN device — a file descriptor the kernel hands IP packets through. The daemon's entire life is one poll over those two fds: UDP readable → decrypt, write to the TUN; TUN readable → encrypt, sendto the peer. Lesson 12 opens that second fd. The loop you just wrote is the one it runs in.

💡

poll scans the whole array every call. epoll does not. With ten thousand connections, copying ten thousand pollfds into the kernel and scanning them back is the cost that dominates. Linux's epoll and BSD's kqueue keep the interest list inside the kernel and return only the ready fds. Same model — a loop over readiness — different data structure. Learn poll first; everything about epoll is the same idea with a registration step.

one-line takeaway

An event loop never blocks on a socket — it blocks in poll, which wakes when any fd is ready, then touches only those fds with non-blocking calls that treat EAGAIN as "not yet".