How Does The FIX Protocol Work?
In other posts in this series, I’ve introduced some of the key concepts of the FIX protocol, as well as some of the history. In this article, I’d like to dive into the basics of how the protocol itself works.
What is a FIX Engine? Where Does It Sit?
First of all, we need to talk about an important piece of software called a FIX engine. Each party would have one or more trading applications, which would interface to a FIX engine, which in turn connects over a network to others. So you can think of a FIX engine as the computer component that is the real interface between two parties who are talking FIX. Multiple trading applications can share a single FIX engine if you wish, and a single FIX engine can connect to multiple counterparties.
The FIX engine may either be a commercial piece of software, such as those licensed by Itiviti, B2Bits or FIX Flyer or it might be an open-source library such as QuickFIX or Fix8. There are FIX engines in most major programming languages.
The network in this regard might be the public internet (typically via a secure stunnel or VPN), or a point-to-point telecom provider such as IPC, or BT Radianz, or a hub-and-spoke FIX network such as NYFIX. The choice of network doesn’t impact the way that FIX messages are constructed or read, however, it is simply the piece of wire between two FIX engines.
Anatomy Of A FIX Message Body
The core role of the FIX engine is to convert an instruction from the trading application into and out of a FIX message suitable for sending over the network. It is therefore a translation engine of sorts.
The most common way in which a FIX message is sent over a network is as a tag-value string. [Note that the standards body (FIX Trading Community) have defined other “encodings”, but tag-value remains the most common. Click this link to learn about other encodings]
To build a tag-value string, we first construct a series of “pairs” of the format <tag>=<value> to represent the information that we want to convey. The tag here is a number defined by the standards body to represent a specific piece of data (you can look them up using a tool such as Fiximate). The value is the value of that particular field that you would like to communicate to the other side. This might be a free-form string, or it might be one of a pre-defined set of enumerations defined by the standards body.
So for example, let’s assume that I wanted to trade some shares of Apple Computers. Then I know that the exchange refers to Apple shares using the code ‘AAPL’ and that the FIX Protocol defines a tag 55 to refer to stock codes. And so the tag-value pair for this element of the message would therefore be:
55=AAPL
Of course, a single message will contain lots of informational elements like this, including whether I’d like to buy or sell, the number of shares to trade and the price at which I’d be willing to do this etc. And so the task of the FIX engine (and the developer integrating the application into it), is to map this information into the correct FIX tag-value pairs.
One really important point to note here, however, is that each API is different. The FIX Trading Community define the standards — a massive superset dictionary of possible fields and values — but each API will only use a subset of this total universe (and can even add extra fields if they need to). For this reason, it is really important to get the documentation for the specific API you are connecting to and understand their unique “flavour” of the protocol rather than assuming that they accept the “full” protocol.
OK, so we now have a series of tag-value pairs which represent our message to the counterparty. These need to be joined together into a single string using the SOH (ASCII 1) character. Now, SOH is a special, unprintable control character which makes it quite tricky to represent on paper, which is why people often replace it with a character such as the vertical pipe (|) character to make it more obvious, and I’m going to adopt the same approach for the rest of this article.
As a general rule, the order in which tags appear within the string does not matter except for repeating groups, which I’ll cover in the section below.
So once we have our concatenated string of instructions, then we should have the body of our message a little bit like this:
1=1|11=9|22=8|38=666|40=2|44=88|54=1|55=AAPL|59=0|60=20210526–14:10:16.403
FIX Header & Trailer
Before we can send our FIX message over the network, however, we must wrap it in an envelope by adding a header and trailer to it. These header and trailer blocks (or “components”) have a common structure for all FIX messages across the API, and therefore they are typically defined separately in the API documentation.
The header and trailer perform several important functions related to the operation of the FIX connection itself.
Addressing
The header contains a series of standard tags describing who is sending the message (tag 49 or SenderCompID), and who the intended recipient is (tag 56 or TargetCompID). These are short “names” that both parties agree will be their identifiers in this connection. They aren’t centrally allocated or registered — the parties can agree to anything they want — and they don’t change over the lifetime of the connection, which explains why you often see CompIDs referring to company names that have since been taken over or gone out of business.
FIX also defines a series of additional identifier for even more granularity of where the instruction came from (or should be sent to), but that is beyond the scope of this article.
Protocol and Message Identification
The very first tag in every FIX message is 8 (BeginString), which indicates the protocol version itself (eg 8=FIX.4.4). And the third tag in every FIX message is 35 (MsgType) which indicates the type of message being sent. For example, a Logon message is indicated by an upper-case A value, and so would include 35=A.
Together, these values are used by the receiving FIX engine to begin understanding the message it has just received and validate it accordingly.
Message Sequence Number (Missed Messages)
The header will also include a sequence number value in tag 34 (MsgSeqNum), which is used to detect whether a message has been lost somewhere.
The process is simple. Imagine that, at the start of each day, both FIX engines start with a sequence number of 1, and every time they send a message out, they increase this counter by one. So the second message contains 34=2 and so on. Then if the receiving FIX engine can read this tag 34 and know what the expected next message will contain in this field. If the next message doesn’t match that expected value, then a message might have been lost and so it initiates a process to recover (re-request) messages from the sending FIX engine.
Message Integrity
Each FIX message also contains two tags designed to check message integrity.
The first is tag 9 (BodyLength) which is always the second tag to appear in the message. This tag contains the expected message length by simply counting the number of characters starting at tag 35 (always the third tag in the message), all the way up to (but not including) tag 10 (always the last tag in the message). Note that the SOH delimiters are counted as a single character in this calculation.
The second integrity tag is 10 (Checksum), which is always the last tag to appear in the message. It is calculated by summing the ASCII character value of all characters in the message (including everything in the header but excluding the CheckSum field itself), performing modulo 256 calculation on that total and then padding the value with zeros to make sure it is three characters long.
If these calculations sound complex, then don’t worry — FIX engines will typically do this stuff for you. It is worth noting, however, that both the BodyLength and Checksum values are calculated across the header, body and trailer portions of the message, and so it is important to glue it all together before calculating these numbers.
Repeating Groups
Another important construction in FIX messages is the idea of a repeating group. This is a structure that is used whenever you might need to repeat the same elements (tags) multiple times.
For example, suppose you needed to convey something special such as the identity of two firms (such as the buyer and seller) in a single message, then you might use a repeating group to identify a firm twice — repeating once for the buyer and once for the seller.
Repeating groups should start with a tag which indicates the number of times that a group of tags will repeat (so two in our example). The FIX Protocol has a special datatype for this, known as NumInGroup. [NOTE: this datatype was introduced in FIX 4.4 and so you might see simple integer fields in FIX 4.2 specifications].
Tags with a NumInGroup datatype mark the start of a repeating group, but there is no explicit end marker either the end of the repeating group or each iteration. It is for this reason that the order of tags really matters within repeating groups; you should aim to replicate the order in which fields appear in the API documentation.
This article has presented a high-level introduction to the anatomy of a FIX message, and how it is constructed. As you can probably see, there is a lot of detail in why FIX is as it is. The good news is that FIX engines handle a lot of this for the developer, but a good working knowledge of how FIX messages are interpreted is useful for roles such as QA, production support or onboarding where staff often have access to raw FIX messages such as this.
Want to learn more about FIX from experienced tutors in a small, online, interactive group? Check out our training at https://fixspec.com/training.
I will continue to post more educational articles on FIX on this channel, so be sure to follow the channel to be notified of updates. We also have a YouTube channel on which we talk about FIX protocol and efficient processes, and you can also follow us on X (@fixspec).
