Hikikomori

21 object(s)
 

605. networking

Networking

At its core, networking is just tagging data with the right addresses and letting the network deliver it. The concept is one sentence, but the failure modes fill textbooks: packets arrive out of order, links congest, addresses exhaust, adversaries forge labels. The layered protocol stack and the protocols within each layer exist to solve these problems reliably at scale.

I

1.1. Switching & Multiplexing

Given a network is a graph of nodes joined by links (e.g. the copper pair, the fibre strand) that exchange data under shared protocols, many sources contending for finite link capacity poses two problems, i) path selection: the route by which data traverses intermediate nodes from source (src.) to destination (dst.); and ii) link sharing: how multiple transmissions divide a single physical link. Switching and multiplexing address them respectively, while the latter partitions a link of capacity $C$ among competing flows, and its discipline distinguishes the switching methods and governs the resulting throughput, latency, and loss.

Specifically, given that multiplexing admits i) time-division multiplexing (TDM): each flow transmits in a fixed time slice; ii) frequency-division multiplexing (FDM): each flow occupies a separate frequency band; and iii) statistical multiplexing: flows share capacity on-demand without pre-allocation; circuit switching leverages TDM/FDM and reserves resources (e.g. bandwidth) for the connection’s lifetime irrespective of utilisations. For example, the public switched telephone network (PSTN), where each call reserves a dedicated circuit, suits steady voice calls though it reveals the approach’s fundamental inefficiency for bursty, short-lived computer traffic.

Packet switching instead adopts statistical multiplexing, partitioning data into packets that share the link on-demand for near-maximum utilisation. Each packet carries its own destination address, and a store-and-forward packet switch forwards it hop-by-hop by matching that address against a lookup table to select the next hop, a role realised at multiple layers. The per-packet latency can be decomposed as $d_{\text{total}} = \sum_{i=1}^{N}(d_{\text{proc}} + d_{\text{queue}} + d_{\text{trans}} + d_{\text{prop}})_i$, where $N$ is the number of hops, $d_{\text{trans}} = L/R$ (i.e. push $L$ bits onto the link at rate $R$), and $d_{\text{prop}} = d/s$ (i.e. signal traverses link of length $d$ at speed $s$), whilst $d_{\text{proc}}$ (e.g. header lookup, checksum) and $d_{\text{queue}}$ (e.g waiting behind other packets) are non-deterministic.

1.2. Network Models

Networks are also classified by geographic scope, local area network (LAN) $\subset$ metropolitan area network (MAN) $\subset$ wide area network (WAN). A LAN spans a building/campus at high bandwidth and sub-millisecond latency, predominantly over Ethernet (1973, standardised as IEEE 802.3). A MAN spans a city, a WAN spans countries or continents, interconnecting many LANs. Over WAN distances, $d_{\text{prop}} = d/s$ dominates, fixed by distance and signal speed, hence irreducible by bandwidth (a transatlantic round trip costs tens of ms), whereas LAN latency is transmission-bound. ARPANET (1969), the first packet-switched WAN, grew into the foundation of the modern internet.

The open systems interconnection (OSI) model, set by international organisation for standardisation (ISO, 1977-84), has seven layers with strict separation of concerns. Designed by committee ahead of implementation, its protocol suites (e.g. X.25 for packet switching, X.400 for email, X.500 for directory services) proved over-specified and costly to implement. TCP/IP however took the opposite approach: implement and standardise what works. As a consequence, the US government also abandoned its OSI mandate (GOSIP, 1990) by 1995. Though OSI lost as a protocol suite, its layer numbering persists as standard vocabulary.

The transmission control protocol / internet protocol (TCP/IP) model was developed through running implementations on ARPANET (NCP → TCP/IP on 1 January 1983, Flag Day), prevailing by shipping working code before any complete specification. It stacks four layers: i) link: frame transmission over a physical medium; ii) internet: logical addressing and best-effort routing; iii) transport: end-to-end process-to-process communication; and iv) application: user-space service semantics. This embodies the end-to-end principle (1984), which holds that application-specific functions should reside at the endpoints rather than in the network, with the two middle layers implemented in kernel space.

Each layer encapsulates the unit above as opaque payload and prepends only its own header, producing its protocol data unit (PDU), e.g. { $\text{L}1$: bits on the wire, $\text{L}2$: a frame, $\text{L}3$: a packet, $\text{L}4$: a segment (TCP) or datagram (UDP), $\text{L}7$: a message or data}. The receiver decapsulates this in reverse by stripping headers layer-by-layer. The isolation lets each layer evolve independently, so Wi-Fi replaces Ethernet at $\text{L}2$, or IPv6 replaces IPv4 at $\text{L}3$ without disturbing the others. Since forwarding is hop-by-hop at $\text{L}2$ but end-to-end at $\text{L}3$, MAC addresses change at every hop (local) while IP addresses stay constant (global), and ports identify the destination process ($\text{L}4$).

1.3. Ethernet [L1/L2]

Ethernet spans the physical and link layers, partitioning the work between them. At $\text{L}2$, the media access control (MAC) sublayer frames the payload by prepending src. and dst. MAC addresses and also appending a CRC-32 checksum for error detection, where each MAC is a 48-bit identifier assigned to a network interface card (NIC) on a host device. At $\text{L}1$, the physical sublayer (PHY) serialises this frame onto the medium (i.e. cable) through a line code that fixes voltage, clocking, and modulation (e.g. 100BASE-TX: 4B/5B, 1000BASE-X: 8b/10b). The frame is thus the $\text{L}2$ unit and the encoded bitstream the $\text{L}1$ unit, both governed by one standard.

A MAC serves as a delivery address only within a single broadcast domain (i.e. the nodes reachable by one $\text{L}2$ broadcast), so link-layer switches forward frames within that domain by MAC (from a forwarding table built by self-learning, IEEE 802.1D). The sender must frame to the next hop yet knows only its IP, a gap the address resolution protocol (ARP) bridges by broadcasting a request to ff:ff:ff:ff:ff:ff that the owner answers and the sender caches. When the dst. IP falls outside the local subnet (determined by the subnet mask), the sender resolves the router’s MAC instead and frames the packet to that gateway, which forwards it onward at $\text{L}3$.

A frame’s payload is bounded by the maximum transmission unit (MTU), the largest L3 packet a link carries (1500 bytes on Ethernet), exclusive of the L2 header and CRC, so the on-wire frame is larger still. Headers eat into this budget, leaving $\text{IP packet} \leq \text{MTU}$ and $\text{payload}_{\text{L4}} \leq \text{MTU} - H_{\text{IP}} - H_{\text{L4}}$ (1460 bytes for TCP over IPv4). Across mixed-MTU links the smallest governs, so an oversized packet is fragmented or dropped unless path MTU discovery probes that minimum (IPv4 DF bit plus an ICMP reply) and the sender pre-sizes to fit. Data-centre and GPU-cluster fabrics raise it to jumbo frames (9000 bytes), amortising header overhead across distributed-training transfers.

II

2.1. IP [L3]

While a MAC delivers only within one broadcast domain, internet protocol (IP) provides the $\text{L}3$ abstraction that spans heterogeneous physical networks (e.g. differ in frame format $A \neq B \neq C$ and addressing $A = B \neq C$, where $A$: Ethernet, $B$: Wi-Fi, $C$: cellular), unifying them under a single logical address space with best-effort delivery. An IP address is a fixed-length identifier, either 32-bit IPv4 (RFC 791, 1981) or 128-bit IPv6 (RFC 2460, 1998), standardised by the IETF in requests for comments (RFCs). Such networks joined by routers form an internet (inter-network), and the Internet denotes the global instance, grown from ARPANET, reaching 6 billion users (74%, ITU 2025).

Every packet leaving its subnet passes through a router, the $\text{L}3$ realisation of the packet switch, stitching heterogeneous $\text{L}2$ networks into one internet. A link-layer switch stays within one broadcast domain, whereas a router forwards between domains, so a host reaches anything off-subnet only through its default gateway. The router strips the incoming $\text{L}2$ frame, reads the destination IP, looks up the next hop, decrements the time-to-live (TTL), and re-frames to that hop’s MAC, the per-packet data plane distinct from the control plane that builds the table. A router thus forwards toward a destination network, not an individual host.

Routing cannot scale to $2^{32}$ individual hosts, so IP splits each address into a network prefix and a host part, collapsing many hosts into one routable network, and only the destination network resolves the individual host. Classful addressing (RFC 791, 1981) gave this split its first form, fixing the prefix length $n$ at 8, 16, or 24 by an address’s leading bits, so Class A, B, and C networks held about 16M, 65k, and 254 hosts. The gaps were coarse, so an organisation needing 500 hosts overflowed a Class C and took a whole Class B (/16), stranding 65,000 addresses on one flat broadcast domain.

Subnetting (RFC 950, 1985) freed the boundary to any bit through the subnet mask, $n$ leading ones a host ANDs with an address to recover its prefix, each added bit doubling the subnets and halving the hosts. A subnet is thus the set of addresses sharing a prefix, one broadcast domain of $2^{32-n} - 2$ usable addresses; two hosts reach each other directly at $\text{L}2$ if and only if their prefixes coincide, else via an $\text{L}3$ router. Classless inter-domain routing (CIDR, RFC 1519, 1993) then abolished classes internet-wide, so $n$ can take any length and adjacent prefixes aggregate into one route, curbing forwarding-table growth.

IPv4’s $2^{32}$ addresses exhausted even under CIDR. Address allocation for private internets (RFC 1918) reserves three non-routable ranges {10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16}, which network address translation (NAT), specifically port address translation (PAT), multiplexes behind one public IP by rewriting source ports, making them reusable across millions of networks but breaking end-to-end addressability. An address thus sits at one of three scopes, loopback (127.0.0.0/8) $\to$ private (a LAN) $\xrightarrow{\text{NAT}}$ public, and IPv6’s $2^{128}$ space restores end-to-end reachability, largely retiring NAT as the dual-stack transition proceeds.

The same prefix that decides locality also drives delivery. The next-hop lookup is a longest-prefix match, the most specific routing table entry containing the destination, with a default route (0.0.0.0/0) as fallback, and routing protocols build that table. Within an autonomous system (AS), one operator’s network, intradomain routing seeks shortest paths, open shortest path first (OSPF) running Dijkstra on a link-state map and routing information protocol (RIP) exchanging distance vectors by Bellman-Ford. Between ASes, interdomain routing by border gateway protocol (BGP) advertises prefix reachability and picks paths by operator policy and AS-path length rather than distance, knitting the autonomous systems into one Internet.

2.2. TCP [L4]

Given that IP is not responsible for reliability, transmission control protocol (TCP, RFC 675 1974; RFC 793 1981, now RFC 9293 2022) provides reliable, ordered, byte-stream delivery over an unreliable packet-switched network. A TCP connection is identified by a 4-tuple (src IP, src port, dst IP, dst port), denoted IP:port (e.g. 127.0.0.1:8080), where ports are 16-bit integers (0-1023 reserved) that multiplex a single IP to distinct processes.

The three-way handshake (3WHS) establishes a connection: the client sends SYN with initial sequence number (ISN) $x$, the server replies SYN-ACK with ISN $y$ and ACK $x+1$, and the client completes with ACK $y+1$, costing one RTT before data transfer. Teardown uses a four-way FIN exchange. To prevent stale segments from a previous connection being accepted, the closing side enters TIME_WAIT for $2 \times$ MSL (maximum segment lifetime), so 240s under RFC 793’s 120s MSL though Linux caps TIME_WAIT at 60s. Each connection also begins with a distinct ISN, originally a 32-bit counter ticking every ~4µs (RFC 793) to separate incarnations, now a cryptographic hash (RFC 6528) that also resists off-path sequence prediction.

Reliability is achieved through sequence numbers and acknowledgements. Each byte in the stream is numbered, and the receiver sends cumulative ACKs indicating the next expected byte. The sender maintains a sliding window of unacknowledged data and retransmits segments not acknowledged within the retransmission timeout (RTO), dynamically estimated from smoothed RTT via Jacobson’s algorithm. Fast retransmit triggers on three duplicate ACKs for the same sequence number, inferring loss and retransmitting immediately without waiting for RTO, reducing recovery from hundreds of milliseconds to roughly one RTT.

Flow control and congestion control are distinct mechanisms. Flow control prevents the sender from overwhelming a slow receiver via a receiver-advertised window (rwnd); congestion control prevents overwhelming the network via a congestion window (cwnd). The sender transmits at $\min(\text{rwnd}, \text{cwnd})$, so the tighter constraint always governs throughput. Congestion control was absent from the original TCP; it was introduced after the 1986 internet congestion collapse, formalised by Van Jacobson in 1988. Slow start initialises cwnd $= 1$ maximum segment size (MSS, though modern stacks start at 10 per RFC 6928) and doubles it each RTT until reaching threshold ssthresh. Beyond ssthresh, congestion avoidance applies additive increase/multiplicative decrease (AIMD), incrementing cwnd by one MSS per RTT and halving on loss.

A socket (BSD 4.2, 1983) is a communication endpoint exposed as a file descriptor, created by specifying an address family and a socket type. The address family determines what the socket is bound to: AF_UNIX for a local filesystem path (§603#3.1, UDS), AF_INET for an (IP, port) pair. The socket type determines the delivery semantics: SOCK_STREAM for reliable byte streams (TCP), SOCK_DGRAM for best-effort datagrams (UDP). BSD 4.2 unified local and network IPC under this single API, extending the fd interface (§603#3.3) from pipes and files to network endpoints, so the socket became the common gateway for both local IPC and network communication. Sockets are a message-passing mechanism (§603#3.1) regardless of address family, as data is always copied through the kernel rather than shared directly. The application code is nearly identical in both cases; only the address struct differs, and the network stack transparently handles routing and reliability when the two processes happen to be on different machines.

A TCP server creates a socket, calls bind() to attach it to a local (IP, port), then listen() to mark it as a passive socket that accepts incoming connections. Each accept() call blocks until a client connects via the three-way handshake, then returns a new connected fd for that client’s 4-tuple, which is why §604#1.1’s thread-per-connection model spawns one thread per accept(). Higher-level libraries (e.g. Python’s requests) abstract this entirely, calling down through urllib3 → socket module → POSIX socket API → kernel TCP/IP stack.

2.3. UDP [L4]

Where TCP layers reliability, ordering, and flow control over IP, user datagram protocol (RFC 768, 1980) keeps IP’s best-effort, connectionless delivery and adds almost nothing but ports and a checksum. Its header is only 8 bytes (source port, destination port, length, checksum), compared to TCP’s minimum 20-byte header, and there is no handshake, no acknowledgement, no retransmission, no ordering guarantee, and no flow or congestion control. This simplicity is a deliberate design choice for applications where the overhead of reliability exceeds the cost of occasional data loss. Like TCP, UDP is accessed through the socket API, using SOCK_DGRAM instead of SOCK_STREAM.

DNS queries use UDP because the typical request-response exchange fits in a single datagram, and the one RTT saved by skipping TCP’s three-way handshake matters for a service invoked before every HTTP connection. Real-time voice and video (VoIP, video conferencing) prefer UDP because retransmitting a stale audio frame that arrives 200 ms late is worse than simply dropping it and continuing with fresh data. Online games send frequent state updates (player positions, actions) at 20-60 Hz, where each update supersedes the last, making reliability for individual packets unnecessary. In each case, the application either tolerates loss, implements its own selective reliability on top of UDP, or treats the latest packet as the only one that matters.

Without built-in congestion control, a UDP application that sends at full rate regardless of network conditions risks contributing to congestion collapse, where the network is saturated with retransmissions and useful throughput approaches zero. Well-behaved UDP applications implement their own rate limiting or congestion-aware pacing. Packets exceeding the path MTU are fragmented at the IP layer in IPv4, and since losing any one fragment forces retransmitting the whole datagram, a UDP app keeps each message within the MTU, which is why DNS falls back to TCP when a UDP response is truncated, historically above 512 bytes though EDNS(0) now raises that ceiling to ~1232 bytes.

III

3.1. DNS (+ CDN) [L7]

Transport carries bytes between numeric addresses, leaving the application layer to define what they mean, beginning with the service that turns a name into an address. Domain name system (RFC 1035) (DNS, 1987) is a hierarchical, distributed system that resolves domain names to typed records (i.e. conventionally port 53). A query such as dig google.com A (or nslookup) triggers a recursive resolver (e.g. ISP-provided) to resolve the domain proceeding through: i) 13 root server clusters identify the top-level domain (TLD) where anycast routes a query to the nearest physical instance sharing the same IP; ii) the TLD server delegates to the domain’s nameserver; and iii) the authoritative nameserver provides an A record (IPv4); Each response carries a TTL governing how long resolvers may cache this.

DNS record types extend beyond A records, including AAAA (IPv6), CNAME (alias), MX (email routing), TXT (domain verification), and SRV (service host, port, priority). In particular, CNAME records enable content delivery networks (CDNs), where a domain’s CNAME redirects resolution to the CDN’s namespace and anycast returns the nearest point of presence (PoP). Providers such as Cloudflare (~330 cities), Akamai (~4,100 PoPs), and CloudFront (~750 PoPs) operate using HTTP caching headers (Cache-Control, ETag) to govern freshness across them, while the PoP serves cached content directly and falls back to the origin on a cache miss.

Without encryption or signing, DNS is vulnerable to cache poisoning (i.e. forged records injected into a resolver’s cache). DNS security extensions (DNSSEC) added cryptographic signatures for integrity, while DoH (DNS over HTTPS) and DoT (DNS over TLS) encrypt queries for confidentiality.

3.2. HTTP [L7]

By 1989 the internet was two decades old, yet the two capabilities a web of documents would need still sat in separate systems, i) reach: crossing to a document on another machine; and ii) linking: an inline jump to another document. Reach existed without linking. FTP (1971) and email over SMTP (1982) moved documents across machines as flat files, no links. Linking existed without reach. It began with the Memex (1945) in As We May Think , a microfilm desk whose associative trails linked documents. Apple’s HyperCard (1987) made hypertext mainstream, but its links never left one machine, unable to name documents elsewhere. No system held both.

At CERN, physicists on incompatible machines lost documents across unlinked systems. Tim Berners-Lee’s 1989 memo Information Management: A Proposal joined the two halves with three pieces, i) hypertext markup language (HTML): a document format carrying inline links; ii) uniform resource locators (URLs): one scheme naming any document on any host; and iii) hypertext transfer protocol (HTTP): the protocol transferring it; The result, the world wide web (WWW, W3), is an $\text{L}7$ space of linked documents addressed by URL, one application over the internet, not the internet itself. So both are webs, the internet of networks and the web of documents.

While HTML structures a document into headings, paragraphs, and lists, its anchor element places a link anywhere in the running text. Any page can then reference any other on any host, forming a graph rather than a tree. The first browser to render it, WorldWideWeb editor (later renamed as Nexus) on a NeXT machine (1990), even displayed images inline, but only through the NeXT’s own text system, so that ability stayed bound to the platform and was lost once the browser was ported to others. Markdown (2004), a lighter markup language John Gruber designed, compiles to this same HTML while remaining readable as unrendered plain text.

A rival briefly looked likelier to win. Gopher (1991, University of Minnesota) appeared just after the web, was likewise networked, and presented documents through nested menus, spreading faster for a year or two. The two differed not in protocol, since gopher and HTTP are near-twin request-response protocols, but in document model. Gopher stored documents as plain text and confined links to the menus, giving it the tree structure of a filesystem (§603#3.3) or code repository, navigated by drilling down and back up. Its native clients spoke only gopher://, never the web’s http://.

The web soon overtook its rival on two fronts. In 1993 Minnesota’s licensing fees drove adoption off Gopher while CERN placed the web in the public domain, and Mosaic rendered inline images on commodity hardware, a medium richer than text-only Gopher that Netscape (1994) carried to the mass market. Because a URL’s scheme names the protocol, these browsers dispatched ftp://, mailto:, and gopher:// from one address bar, subsuming the rival rather than merely beating it. A fragment of Gopher endures there nonetheless, as Mark McCahill, one of its Minnesota creators, co-authored the URL specification (RFC 1738, 1994) and is credited with coining the term.

The same inline links soon became the web’s retrieval problem. As the corpus grew to millions of documents, the constraint shifted from reaching a page to identifying the relevant one. Curated directories like Yahoo and keyword engines like AltaVista addressed this poorly, the former bounded by human editors, the latter gamed by repeating terms on a page. Google’s PageRank (1998) instead exploited the hyperlink graph, weighting each inbound link by the rank of its source, equivalently the stationary distribution of a random walk over it. The link an author placed for a reader thus doubled as the signal a machine ranked by, so one structure served both navigation and retrieval.

An HTTP request opens in plain text with a start line naming the method, target URL, and protocol version, then header lines of key-value metadata and an optional body, and the response mirrors it with a status line, headers, and body. The target decomposes as scheme://host:port/path?query, its host resolved through DNS, a framing readable enough to inspect directly with curl or telnet. Headers form the protocol’s extensibility surface, the mandatory Host letting one IP serve many domains through virtual hosting, Accept and Content-Type driving content negotiation over the representation and its media type (MIME, RFC 6838), and Accept-Encoding selecting compression (gzip, brotli).

HTTP is stateless by design, the server retaining no memory between requests, so applications must carry state explicitly. Cookies carry a session ID, while bearer tokens like a JSON web token (JWT) ride the Authorization header, both threading client state. The session ID indexes server-side state, whereas a JWT carries signed claims inline, sparing the lookup but resisting revocation. Status codes partition responses into five classes by the leading digit (1xx informational, 2xx success, 3xx redirection, 4xx client error, 5xx server error), of which 200, 301, 404, and 503 dominate debugging.

HTTP/1.0 (RFC 1945, 1996) opened a new TCP connection per request, wasting a round-trip. HTTP/1.1 (RFC 2068, 1997) added persistent connections but still suffered head-of-line blocking, one slow response stalling those behind it. HTTP/2 (RFC 7540, 2015) multiplexed streams over one TCP connection, yet a lone lost segment stalled every stream at once (transport-layer head-of-line blocking). HTTP/3 (RFC 9114, 2022) dissolved this with QUIC (Google, 2012, standardised 2021 as RFC 9000), reimplementing TCP’s reliability and TLS 1.3 encryption over UDP for 1-RTT establishment (0-RTT on resumption). The progression tracks protocol ossification, TCP so embedded in middleboxes that it proved immutable, forcing innovation onto UDP.

Applications sometimes require server-initiated data, but HTTP/1.x is strictly request-response, with the client initiating every exchange and only one in flight at a time per connection. Long polling (~2006, Comet) simulated server push by holding requests open, wasting one TCP connection per pending event. Server-sent events (SSE, W3C 2015) solved it with persistent unidirectional streaming over a single HTTP connection (e.g. LLM token streaming, where tokens are generated sequentially but delivered in chunks due to I/O buffering). WebSocket (RFC 6455) (2011), by contrast, uses an HTTP Upgrade handshake to switch to a full-duplex protocol over TCP, which supports bidirectional use cases (e.g. Slack, Figma).

3.3. HTTPS [L7]

HTTP transmits plaintext, exposing traffic to any observer on the path. Transport layer security (TLS), evolved from Netscape’s Secure Sockets Layer (SSL, 1995, now obsolete), provides confidentiality, authentication, and integrity over TCP, and HTTPS is simply HTTP spoken inside a TLS tunnel (i.e. port 443), though TLS also secures other protocols such as SMTP, database connections, and gRPC. In TLS 1.3 (RFC 8446, 2018), the client and server exchange cipher suite preferences and ECDHE key shares in a single round-trip, deriving a shared secret and verifying the server’s identity via its X.509 certificate. Resumed connections support 0-RTT by reusing a cached pre-shared key, trading replay vulnerability for zero-latency establishment.

The server’s identity is verified through a certificate chain of trust: the server presents a certificate signed by an intermediate CA (Certificate Authority), itself signed by a root CA pre-installed in the client’s trust store. The client validates each signature and checks hostname match. TLS further achieves forward secrecy through ephemeral key exchange, where each session generates a unique Diffie-Hellman key pair discarded after the handshake, so a compromised long-term key cannot decrypt past traffic. This is why TLS 1.3 mandates ephemeral exchange and removes static RSA.

IV

4.1. Application Programming Interface

At $\text{L}7$ the payload arrives intact but uninterpreted, what its bytes mean fixed by the two endpoint programs alone. An application programming interface (API) is that contract, one program’s operations exposed to another, from an in-process call over shared memory to a networked service serialised across a boundary. That boundary fixes the contract along two axes graded by coupling, the prior agreement the two ends must share before they interact, i) format: the data’s encoding, which either side may swap unannounced, hence interchangeable; and ii) style: the operations and their semantics, pinned in both ends in advance, hence architecturally decisive.

Formats trade legibility for density. Text formats such as JavaScript object notation (JSON) and extensible markup language (XML) stay human-readable and self-describing but verbose and slow to parse; binary formats such as protobuf are compact and fast yet opaque, their fields written straight to bytes. A schema’d format binds both ends to one definition, the coupling the styles trade on, REST’s schemaless JSON against gRPC’s schema-bound protobuf. YAML, a superset of JSON, stays a config and spec format (OpenAPI included), never a wire one, ruled out by significant whitespace and unsafe deserialisation.

The split between text and binary follows from how serialisation works. A pointer is valid only in the address space that created it, where data lives as pointer-linked objects (§603#3.2), while the channel, a socket, pipe, or file, carries a flat byte sequence, not a heap. To deliver a message the sender serialises its object to text, then encodes that text as 8-bit unicode transformation format (UTF-8) bytes, which the receiver decodes and deserialises back into an equal object; a binary format collapses the two, its fields written to bytes directly. Those bytes are canonical and platform-neutral, read identically whatever the two ends’ language, endianness, word size, and float representation (§601#1.2). This encoding, not delivery, is OSI’s presentation layer ($\text{L}6$), folded into the application by TCP/IP.

The format fixes not just how bytes read but how they may change, the harder half once clients depend on the contract and cannot be redeployed. A change must stay backward-compatible, an old client still understood by a new server, and forward-compatible, a new client tolerated by an old server, the contract hardened into a narrow waist. Most of this is schema evolution, additive, old fields never removed or repurposed, catalogued in Kleppmann and bound more or less firmly by format, protobuf through immutable field numbers and reserved tags, JSON through a tolerant reader, machine-checkable where schemaless JSON is left to discipline.

Only when a change cannot stay compatible is the API versioned, by URL (/v2/), header, or media type, each trading discoverability against cache and routing cleanliness. Stripe pins a version per account and threads old requests through compatibility shims, one code-base serving a decade of clients, the approach modern guides such as Google’s AIP favour over proliferating URLs, retirement announced through a deprecation window and a Sunset header (RFC 8594). Format thus settled, the remaining sections turn to the other axis, style, graded by coupling, fixed by who the two ends are and whether they deploy together, an unknown public client the server cannot redeploy forcing the loosest contract while two co-deployed services afford the tightest. Coupling does not track the calendar, the tight procedure contracts of RPC and SOAP coming first, REST then loosening to inherit the web, GraphQL and gRPC drawing it back as relationships narrowed, each style the contract a relationship makes possible.

4.2. REST

REST, the most general style, came from the web itself, whose client-server exchange is already an API, one HTTP request handing a document to a browser or data to a program, the same exchange up to representation and consumer. Roy Fielding, who co-authored HTTP/1.0 with Berners-Lee, then shaped HTTP/1.1 and the uniform resource identifier (URI) syntax, abstracted from the running web the model that guided each revision, and named it in his 2000 dissertation representational state transfer (REST), a description of a system already in use. Its gain is generality, one interface reaching any resource a program names, the two ends sharing nothing bespoke beyond HTTP’s generic contract and coupling as loosely as the web.

Parenthetically, REST’s takeover was a contest, not a coronation, won by going with the web’s grain where its rivals cut against it. It rode HTTP alongside the simple object access protocol (SOAP) and XML-RPC (1998), the rivals burying their calls in opaque XML POSTs no intermediary could read, while it named the target in the URL and the effect in the verb, cacheable by method and URL alone. The same divide told in the formats, where lighter, browser-native JSON beat XML’s markup heft, inherited from the standard generalised markup language (SGML) roots it shares with HTML, REST riding the winner while SOAP stayed bound to its XML envelopes.

In REST anything nameable is a resource named by a URI. The server returns not the resource but a representation of its state (e.g. a JSON snapshot). The wire carries no schema, OpenAPI (formerly Swagger) recovering one out of band to generate SDKs, stubs, validation, and docs. A client acts on the resource through HTTP’s own verbs, {create: POST, read: GET, update: PUT, delete: DELETE}, each graded by the guarantee it carries, safety (state unchanged) strictly stronger than idempotence (a retry reproduces the effect), so safety embeds in idempotence, {GET} ⊂ {GET, PUT, DELETE}, while POST holds neither, a repeat duplicating unless an idempotency key collapses it onto the first.

A collection is itself a resource, GET /orders returning its members as a list, unbounded, so the server sends one page at a time, each page’s edge fixed by position or by value. An offset counts rows to skip, the simplest choice yet flawed twice, the database re-scanning the skipped prefix at $O(\text{offset})$ while the count drifts as concurrent writes shift every later row, so records repeat or vanish between pages. A cursor fixes the edge by the last-seen value of the sort key instead, resuming past it through an index at $O(\log n)$ and immune to that drift, at the cost of forbidding jumps to an arbitrary page and demanding a unique ordering to break ties, the scheme Stripe and Google’s AIP-158 settle on.

Beneath that surface sit six constraints, the source of its scalability. Statelessness contributes most, inherited from HTTP and here elevated to a constraint, each request self-describing, hence any server answers any request and capacity scales horizontally in the machine count. The strictest is the uniform interface, its hardest clause hypermedia as the engine of application state (HATEOAS), the client driven by server-returned links rather than URLs fixed in advance, which renders the open web the purest REST while most JSON services hardcode endpoints, keeping the name and dropping the constraint, the decay Fielding decried in 2008.

4.3. GraphQL

A fixed REST endpoint over-fetches a whole resource for one field, or under-fetches into several round-trips. GraphQL (Facebook, 2012 for its mobile clients, open-sourced 2015) answers both by moving the shape of the response from server to client. A single typed schema, the first agreement REST did without, describes the graph of available fields, and one endpoint accepts a query naming exactly the fields wanted across related entities, returning them in one round-trip, each field answered by its resolver, the server-side function fetching that field from its table or service, so the join runs inside the server rather than across the client’s round-trips. The gain costs the leverage REST drew from HTTP, a lone POST endpoint defeating per-URL caching, and a query nesting to depth $d$ over branching factor $b$ fanning out to $O(b^d)$ resolver calls, the $N+1$ problem its routine instance.

Those costs are paid down in practice, dataloader batching collapsing the $N+1$ to $1+1$ by coalescing the keys requested within a tick into one query, while persisted queries (a hash standing in for the query text) restore GET caching and depth and complexity ceilings bound a query’s cost. GraphQL also rarely fronts one service, its schema an aggregation layer, Federation composing subgraphs, each owned by a separate backend, into one supergraph the gateway resolves, every field dispatched to its owning REST, gRPC, or datastore and reassembled into the requested shape, a backend-for-frontend before a microservice fleet rather than between its members. GitHub’s v4 API and Shopify expose their platforms through a single such graph.

4.4. RPC

REST addresses a resource and hands back a snapshot of its state, whereas RPC names a procedure for the server to run, a noun answered by state against a verb answered by a computation. Between co-deployed services REST and GraphQL’s public-edge generality is dead weight, text over HTTP paying for reach no internal caller needs, and remote procedure call (RPC, 1984, formalised by Birrell) tightens here, the client invoking the procedure through a stub, a local proxy generated from an interface definition both ends compile in advance. The stub marshals the arguments and unmarshals the reply so the call reads as local, an abstraction that leaks where the network fails or stalls as a local call would not.

Tunnelled through HTTP by SOAP and XML-RPC, RPC lost to REST at the edge but returns on its own protocol, its implementations spread across coupling. JSON-RPC 2.0 (2010) sits at the loose end, a schemaless, transport-agnostic object of method, params, and id. gRPC (Google, 2015), its internal Stubby opened up, holds the tight end, typed protocol buffers (protobuf) over HTTP/2, whose multiplexing carries unary or streaming calls, fast but cache-opaque and unreachable from a browser without a grpc-web proxy, a fit for intra-cluster links and ML serving.

The language server protocol (LSP, Microsoft, 2016) first built on JSON-RPC at scale. $M$ editors wired to $N$ languages by hand form an $M \times N$ tangle of bespoke plugins, which LSP collapses to $M+N$, one JSON-RPC contract each language implements once for every editor and each editor once for every language. The insight was that language intelligence is a service, not an editor feature, separated from the UI it composed freely, and rich completion and navigation turned from a per-editor luxury into a commodity every client inherits. A language server, a separate process speaking the protocol over stdio or a socket, answers requests for completion, diagnostics, and go-to-definition, and the editor stays ignorant of the language’s toolchain, a template a new $M \times N$ tangle would soon borrow.

Anthropic’s model context protocol (MCP, 2024) applies the same $M+N$ collapse to LLM agents, built on the same JSON-RPC, one open standard a tool implements once for any client, replacing the hand-wired tangle of apps against tools, adopted by rival labs by 2025. A model’s reach is bounded by what it can call, and a shared protocol for tools and context is what turns a chatbot into an agent, its context window a bus any capability can plug into. A server exposes {resources: data, tools: typed functions, prompts: templates} over stdio or HTTP. Its transport evolved like HTTP’s, per-client HTTP+SSE giving way to a stateless Streamable HTTP (2025) endpoint any node can serve, echoing QUIC displacing TCP under HTTP/3.



I gathered words solely for my own purposes without any intention to break the rigour of the subjects.
I also prefer eating corn in spiral .

Code licensed under MIT License