Building a Concurrent TCP Server in Go: Connections, Deadlines, and Graceful Shutdown

TCP makes it possible to build extremely simple network services.

Open a listener. Accept a connection. Read bytes. Process them.

The complexity appears when the server has to do this for many clients at the same time while remaining responsive when clients become slow, disappear unexpectedly, or send data indefinitely.

That was one of the core engineering problems behind Strata-Log.

The project uses raw TCP connections for log ingestion, making connection management one of the foundations of the system.

The Basic TCP Server

At its simplest, a Go TCP server looks like this:

listener, err := net.Listen("tcp", address)
if err != nil {
    return err
}

for {
    conn, err := listener.Accept()
    if err != nil {
        continue
    }

    go handleConnection(conn)
}

The model is attractive because it is easy to understand.

Each connection gets its own execution context.

But real systems need additional controls.

Why Concurrency Matters

A log aggregation server cannot process one connection at a time.

Imagine four clients:

Client A ── slow
Client B ── fast
Client C ── fast
Client D ── fast

If the server handles these sequentially, Client A can delay everyone else.

Concurrency allows independent streams to progress without waiting for unrelated connections.

Go’s goroutines make this model straightforward.

The Problem With Slow Clients

A connection can remain open without producing useful data.

If the server waits indefinitely for input, resources can remain occupied unnecessarily.

This is where deadlines become useful.

Go’s networking package provides:

conn.SetReadDeadline(...)

A read deadline gives the connection an upper bound for how long a blocking read should wait.

This does not mean that every timeout represents a failure.

It simply gives the server a mechanism for detecting connections that have stopped behaving as expected.

Why Deadlines Matter in Strata-Log

A log ingestion service needs to remain available for new streams.

A permanently stalled connection should not be allowed to consume resources indefinitely.

Connection deadlines therefore become part of resource management.

The system can detect inactive connections and decide whether they should continue or be closed.

This makes the server’s behavior more predictable under imperfect network conditions.

Separating Connection Handling From Processing

Another important architectural decision is avoiding a design where the connection handler performs every downstream operation itself.

A simplified pipeline looks like:

TCP Connection
      │
      ▼
Read Incoming Data
      │
      ▼
Dispatch Work
      │
      ▼
Worker Pool
      │
      ▼
Storage

The connection layer focuses on networking.

The worker layer focuses on processing.

The storage layer focuses on persistence.

This separation reduces coupling between components.

Why Worker Pools?

A server can create a goroutine for every connection or every unit of work.

But unlimited concurrency is not the same as useful concurrency.

A worker pool provides a bounded execution model:

Incoming Work
     │
     ▼
┌───────────────┐
│ Worker 1      │
│ Worker 2      │
│ Worker 3      │
│ Worker 4      │
└───────────────┘
     │
     ▼
Storage

The number of workers becomes an explicit configuration and architectural decision.

That makes system behavior easier to reason about.

Backpressure

Once processing becomes asynchronous, another question appears:

What happens when incoming work is faster than workers can process it?

This is the beginning of a backpressure problem.

If the queue is unbounded, memory usage can grow indefinitely.

If the queue is bounded, the system needs a policy for what happens when it fills.

Possible strategies include:

blocking producers;
dropping data;
applying rate limits;
rejecting new work;
increasing processing capacity.

There is no universal answer.

The correct strategy depends on the system’s reliability requirements.

For a log aggregation system, silently dropping data can be particularly undesirable.

Graceful Shutdown

Stopping a TCP server is more complicated than terminating the process.

There may be:

active connections;
pending work;
workers still processing data;
buffered data waiting to be flushed.

A graceful shutdown gives these components an opportunity to finish.

A conceptual sequence is:

Shutdown Signal
      │
      ▼
Stop Accepting
      │
      ▼
Close / Drain Connections
      │
      ▼
Stop New Work
      │
      ▼
Wait for Workers
      │
      ▼
Flush Storage
      │
      ▼
Exit

Strata-Log uses synchronization primitives such as sync.WaitGroup to coordinate this lifecycle.

Why sync.WaitGroup?

A WaitGroup is useful when one part of the program needs to wait for multiple concurrent operations to finish.

For example:

var wg sync.WaitGroup

wg.Add(1)

go func() {
    defer wg.Done()
    processConnection()
}()

wg.Wait()

The important concept is ownership.

The system needs to know which operations must finish before shutdown can safely continue.

Designing for Failure

Network systems operate in environments where failure is normal.

Connections disappear.

Clients crash.

Packets are delayed.

Processes restart.

Storage becomes temporarily unavailable.

A resilient server does not assume perfect conditions.

Instead, it defines how each failure should propagate through the system.

That was one of the important lessons from implementing Strata-Log.

Closing Thoughts

Building a concurrent TCP server is not primarily about writing a loop around Accept().

The difficult engineering work happens around that loop.

Connection lifecycle, deadlines, bounded concurrency, backpressure, and shutdown behavior determine whether the server remains predictable when the workload becomes real.

Strata-Log provided a practical environment for exploring those problems.

The result is a system where networking, processing, and storage have explicit responsibilities rather than being collapsed into one large execution path.