Code

View on GitHub

Address conversions #

See how to convert textual and binary IPv4 addresses back and forth:

make -B addrconv

addrconv.c

./addrconv 192.168.1.100
./addrconv 127.0.0.1
./addrconv 256.0.0.1
./addrconv abrakadabra

Binding addresses #

make -B addrbind

addrbind.c

Bind localhost address:

./addrbind 127.0.0.1 8080

Bind INADDR_ANY:

./addrbind 0.0.0.0 8080

Explicitly bind ephemeral port:

./addrbind 127.0.0.1 0

Network setup #

Set up server independently connected with two clients over two distinct networks:

sudo ./two_clients.sh up

two_clients.sh

graph TD
    subgraph ns_client1 ["Namespace: ns_client1 (LAN Simulation)"]
        direction TB
        lo1((lo
127.0.0.1/8))
        veth_c1[veth_c1
IP: 10.0.1.2/24
Def GW: 10.0.1.1]
    end

    subgraph ns_server ["Namespace: ns_server (Application Server & Router)"]
        direction TB
        lo_s((lo
127.0.0.1/8))
        veth_srv_c1[veth_srv_c1
IP: 10.0.1.1/24]
        veth_srv_c2[veth_srv_c2
IP: 10.0.2.1/24]

        forward[IPv4 Forwarding = ENABLED]
    end

    subgraph ns_client2 ["Namespace: ns_client2 (WAN / Chaos Simulation)"]
        direction TB
        lo2((lo
127.0.0.1/8))
        veth_c2[veth_c2
IP: 10.0.2.2/24
Def GW: 10.0.2.1]
    end

    veth_c1 <-->|VETH Pair 1| veth_srv_c1
    veth_c2 <-->|VETH Pair 2| veth_srv_c2

    classDef namespace fill:#f9f9f9,stroke:#333,stroke-width:2px,stroke-dasharray: 5 5;
    class ns_client1,ns_server,ns_client2 namespace;
    
    classDef interface fill:#e1f5fe,stroke:#0288d1,stroke-width:2px;
    class veth_c1,veth_c2,veth_srv_c1,veth_srv_c2 interface;
    
    classDef loopback fill:#f3e5f5,stroke:#8e24aa,stroke-width:2px,shape:circle;
    class lo1,lo2,lo_s loopback;

    classDef forwarder fill:#e8f5e9,stroke:#4caf50,stroke-width:1px,stroke-dasharray: 2 2;
    class forward forwarder;

Namespace ns_server will act here both, as a routing device between networks 10.0.1.0/24 and10.0.2. 0/24 and as a node running the server application and receiving traffic.

Note clients have a default gateway configured to 10.0.1.1 and 10.0.2.1 respectively.

Verify connectivity:

sudo ip netns exec ns_client1 ping -c 2 10.0.1.1 # Client 1 <-> Server

sudo ip netns exec ns_client2 ping -c 2 10.0.2.1 # Client 2 <-> Server

sudo ip netns exec ns_client1 ping -c 2 10.0.2.2 # Client 1 <-> Client 2

sudo ip netns exec ns_client2 ping -c 2 10.0.1.2 # Client 2 <-> Client 1

User Datagram Protocol #

Basic Usage #

make time_server time_client

time_protocol.h time_server.c time_client.c

Run server bound on INADDR_ANY:8080:

sudo ip netns exec ns_server ./time_server 0.0.0.0 8080

And try to have it serve some requests:

sudo ip netns exec ns_client1 ./time_client 10.0.1.1 8080

sudo ip netns exec ns_client2 ./time_client 10.0.2.1 8080

Capture an incoming packet and inspect them:

sudo ip netns exec ns_server tcpdump -i veth_srv_c1 -n -c 1 -v -XX -w udp.pcap --print udp port 8080

tshark -r udp.pcap -V -x

Binding specific interface #

sudo ip netns exec ns_server ./time_server 10.0.1.1 8080

Observe that client 1 is handled as previously, client 2 requests are not processed.

Note packets from client 2 are reaching the server. The OS net stack drops them and generates back ICMP response:

sudo ip netns exec ns_server tcpdump -i veth_srv_c2 -n

Experiment with binding non-local adresses:

sudo ip netns exec ns_server ./time_server 10.0.3.1 8080

Localhost communication #

Try binding lo address:

sudo ip netns exec ns_server ./time_server 127.0.0.1 8080

sudo ip netns exec ns_server ./time_client 127.0.0.1 8080

Note ns_server used in client invocation. External communication won’t in such a setup.

Port choice #

Try to run multiple servers in parallel and note Address already in use error:

sudo ip netns exec ns_server ./time_server 0.0.0.0 8080

You can easily check with ss (socket statistics) tool system-wide port usage:

sudo ip netns exec ns_server ss -aun

Try running server bound on 10.0.1.1 and 10.0.2.1 in parallel.

Note that in the host namespace, typically binding low ports (< 1024) is not allowed.

./time_server 127.0.0.1 999

In explicitly created namespaces there are no such restrictions.

Packet loss #

Inject random packet drops between server and client 2 simulating weak connection:

sudo ip netns exec ns_client2 tc qdisc add dev veth_c2 root netem loss 40%

Run the server as normal:

sudo ip netns exec ns_server ./time_server 0.0.0.0 8080

Observe client 1 running normally:

sudo ip netns exec ns_client1 ./time_client 10.0.1.1 8080

Check how client 1 behaves:

sudo ip netns exec ns_client2 ./time_client 10.0.2.1 8080

Never assume any single UDP packet gets delivered. Always expect it to be lost!

Rollback to the normal state:

sudo ip netns exec ns_client2 tc qdisc del dev veth_c2 root

Reorderings #

Build and try out the client sending a burst of requests:

make time_burst_client

sudo ip netns exec ns_client2 ./time_burst_client 10.0.2.1 8080

Now let’s simulate another possible network behavior - packet reordering:

sudo ip netns exec ns_client2 tc qdisc add dev veth_c2 root netem delay 500ms 400ms distribution normal reorder 50%

Try running client 2 now and observe server and client logs.

Never assume UDP datagram delivery order!

Rollback to the normal state:

sudo ip netns exec ns_client2 tc qdisc del dev veth_c2 root

Datagram truncation #

Run UDP echo server:

make -B echo_server

echo_server.c

sudo ip netns exec ns_server ./echo_server

Utilize nc as a client which sends a datagram line by line:

sudo ip netns exec ns_client1 nc -u 10.0.1.1 5678

Observe that messages longer than the server’s read buffer size get truncated.

Transmission Control Protocol #

TCP Sink Server #

make byte_sink

byte_sink.c

Setup ss watch before running the server:

sudo ip netns exec ns_server watch -n0.1 ss -tan

Setup tcpdump too:

sudo ip netns exec ns_server tcpdump -i veth_srv_c1 -n tcp port 8080

sudo ip netns exec ns_server ./byte_sink

Try connecting to the server before listen() and observe packet dump:

sudo ip netns exec ns_client1 nc -nv 10.0.1.1 8080

Note the remote server responds with RST segment refusing the connection.

Now let the server call listen(). Check ss output. Re-try to connect:

sudo ip netns exec ns_client1 nc -nv 10.0.1.1 8080

Note:

  • client socket emerged in ss in ESTABLISHED state, it has a peer address set.
  • the whole handshake is completed without any server process intervention.
  • client may even send some data and get it acknowledged

Now, let the server call accept(). Observe nothing changes in ss or tcpdump.

Receive bytes, observe decreasing Rx buffer length in ss.

Close the connection by terminating nc and observe server reading EOF.

Follow-ups:

  • Send some data before server calls accept()
  • Close the connection before server calls accept()
  • Try to connect more clients, while the first one is still connected.

Client application #

Now build and use small client application which reads from stdin and writes everything to an established TCP conneciton.

make simple_nc

simple_nc.c

sudo ip netns exec ns_client1 ./simple_nc 10.0.1.1 8080

Experiment as previously.

Try to populate server Rx buffer and C-d to send FIN fin. Observe server reading EOF.

Follow-ups:

  • Kill the server when it’s Rx buffer is empty. Close the client without sending anything more!
    • Observe server socket going into TIME_WAIT state.
    • Try re-running the server immediately after.
  • Kill the server when it’s Rx buffer is empty, then close the client!
    • Observe that the client cannot distinguish between half-closed and fully-closed stream just by writing.
    • Note the first attempt to send works and the next one kills the client application!
  • Kill the server when it’s Rx buffer is not empty
    • Again see client gets killed with signal on second write attempt.

KV Store #

Now let’s build a simple key-value store server which interatively serves many clients, processing their requests and sending reponses.

make kv_store kv_client

kv_protocol.h kv_store.c kv_client.c

Sniff traffic on port 8085:

sudo ip netns exec ns_server tcpdump -i veth_srv_c1 -n tcp port 8085

sudo ip netns exec ns_server ./kv_store

Run the first client and play with the commands:

sudo ip netns exec ns_client1 ./kv_client 10.0.1.1 8085

Without closing the first client run the second one and attempt to do something:

sudo ip netns exec ns_client2 ./kv_client 10.0.2.1 8085

Observe requests are served only after the first client disconnects.

Follow-ups:

  • Try to execute special fire command
    • Observe server gets killed with SIGPIPE.

Broken networks #

TBD

13 kwietnia 2026
Edit this page