ISO 8583 Is One of the Best System Design Examples We Have

ISO 8583 is a message format standard that supports the exchange of card transactions between acquirers and issuers. It is a generic messaging system used by networks like Visa, Mastercard, and many other regional networks.

I have seen systems use this message format for internal communication too, since the transaction processing core is already capable of handling ISO messages. ISO 8583 defines standard fields and also supports custom fields, so it can be used in many cases.

It is super efficient compared to formats like JSON or XML. It uses a bitmap to indicate whether a field is present or not. No field names on the wire, no closing tags, no whitespace. You read the bitmap, you know exactly which fields follow and in what order.

But the format is only half the story. The more interesting part is the communication pattern the standard assumes, because that is what lets these systems scale.

The design decision: async, not sync

In ISO 8583, the acquiring side sends transactions continuously without waiting for a response. It uses a non-blocking, asynchronous message flow pattern.

This is the decision that everything else follows from. In a high-throughput message processing system, waiting for the response to message N before sending message N+1 creates massive bottlenecks. Your throughput becomes a function of your slowest downstream partner, and you have no way out of it except adding more connections.

The standard avoids this by decoupling the request from the response. A 0100 goes out. A 0110 comes back at some point, on the same socket, possibly after three other responses that were sent later. The correlation is carried in the message itself, not in the connection state.

That is the whole trick. Everything below is what you have to build as an acquirer to live with it.

Matching responses to requests

One major complexity is that responses can come back out of order, or not come back in time (say, 15 seconds).

So the acquirer uses specific ISO 8583 fields to match the response to the original request:

• DE 11 (STAN)
• DE 7 (transmission date and time)
• DE 32 (acquiring institution ID)
• DE 37 (RRN)

STAN alone is not unique across acquirers, so DE 32 is needed to make the match reliable. Get this key wrong and you will eventually approve a transaction against the wrong request, which is a much worse failure than a timeout.

Timeouts and reversals

Two things have to be handled properly, and both of them are about the response you did not get.

Response timeouts. Every message should have an internal timer. If the timer expires before the response arrives, the transaction is marked as timed out and a reversal is sent right away. The issuer may have approved the transaction and the response simply got lost, so you cannot assume that no response means no approval.

Late responses. If a response arrives after the timeout, we have to trigger an auth reversal. The customer has already been told the transaction failed, so the hold on the issuer side has to be released.

Persistent connections

A persistent socket is a big advantage of this pattern. Once the connection is established, it stays open, and network management messages (0800 echo test, answered with 0810) are exchanged on the same pipe whenever there is no activity. This completely avoids the overhead of establishing a connection for every message.

Compare this with HTTPS, where every new connection needs a TCP and SSL handshake before any data is sent. Connection pools help there too.

I saw this firsthand in one of our production incidents. We kept re-establishing the SSL connection with a partner again and again, and we had only a 3-second timeout for them to send the response. We had around 60% timeouts, while the partner was saying their 99th percentile response time was 2 seconds. The repeated handshakes were eating into our 3-second budget before the partner even received the request. Once we enabled pooled connections for this communication, the problem was solved.

The partner was not lying and we were not misreading our own metrics. We were just measuring two different things, and the handshake sat in the gap between them.

The correlator pattern

The socket is full duplex, so the writer and the reader are independent. One path writes requests to the socket, and a separate reader loop pulls bytes off the same socket, frames each message using the length prefix, and parses it. The reader has no idea which request a given response belongs to until it reads the correlation fields.

This is where the pending request registry comes in. The pattern I use is an async request-reply correlator, which turns an asynchronous ISO 8583 conversation into a synchronous-looking call for the caller.

1. Shared pending request registry. A registry keyed by the correlation fields.
2. Register and dispatch. Register in the database and in the registry before dispatch, then write to the service bus queue. You can also write to the socket directly here.
3. Await with timeout. Exit on either the response or the timeout.

In C#:

var pending = new TaskCompletionSource<IsoMessage>(
    TaskCreationOptions.RunContinuationsAsynchronously);

// register before dispatch, never after
_waiters[correlationKey] = pending;
await _registry.RegisterAsync(correlationKey, _instanceId);

await _sender.SendAsync(request);

var completed = await Task.WhenAny(pending.Task, Task.Delay(timeout));
if (completed != pending.Task)
{
    _waiters.TryRemove(correlationKey, out _);
    await _registry.RemoveAsync(correlationKey);
    return TimedOut(request);   // send the reversal from here
}

return await pending.Task;

The reader loop does the other half. It parses the response, builds the same correlation key, finds the waiter, and completes it. If the key is not there, the response is late and the reversal path takes over.

In Golang I used the same idea with channels:

ch := make(chan *IsoMessage, 1)   // buffered, so a late writer never blocks

mu.Lock()                         // lock as the map is not synchronized
waiters[key] = ch                 // first register the response channel to avoid fast responses
mu.Unlock()                       // unlock for others to access

// claim the key so any instance that reads this response knows who is waiting for it
registry.Register(ctx, key, instanceID)

go WriteToSocket(req)             // write as a goroutine so it does not block

select {
case resp := <-ch:
    return resp, nil
case <-time.After(timeout):
    mu.Lock()
    delete(waiters, key)
    mu.Unlock()
    registry.Remove(ctx, key)
    return nil, ErrTimeout   // send the reversal from here
}

Two things matter in both versions. Register before you dispatch, otherwise a fast response can arrive before the entry exists and you will time out a transaction that actually succeeded. And make the response channel buffered, so the reader loop never blocks writing to a channel nobody is listening on anymore.

Running this on more than one instance

The code above works as written when one instance owns the socket. Once you scale out, it breaks in a specific way: the response can come back to an instance that did not send the request. The reader loop parses a perfectly valid 0110, looks in its local map, finds nothing, and treats a good approval as a late response.

This is why the registry has to be shared. The local map still exists, because that is what the waiting goroutine or task is actually blocked on, but the correlation key is registered centrally along with the instance that owns it. When the reader loop on instance B parses a response for a key owned by instance A, it publishes the response to A instead of dropping it. Redis pub/sub on a channel per instance is enough. The DB record is the backstop, so if both instances are gone, recovery still has the pending transaction and can reverse it.

The alternative is sticky connections, where each instance owns its own socket to the partner and only ever reads its own responses. That is simpler and I have shipped it, but it costs you flexibility in the connection pool and it makes a single instance restart more visible to the partner.

Why this is worth studying

ISO 8583 is one of the best examples of a well-designed software system. A single card transaction crosses so many parties and so many hops, and it still comes back secure, fast, and reliable. That is not luck. It works because the standards are defined at the protocol level, so every party in the chain knows exactly how to frame a message, how to correlate a response, and what to do when one goes missing.

The part that is easy to miss is the last one. Most of the design is not about the happy path. It is about the response that never arrives, and making sure every party has the same answer for it. As a software architect or a system design reviewer, this is what the design teaches you, and it is what you start looking for in every design you review.

Forty years later, we are still building systems on top of it. That says something.