How do HFT systems store position and order data?

Question

HFT systems are really sensitive in latency. Which means it can't actually use traditional databases like MySQL, PostgreSQL, or etc. But then, how do we actually persist order and position data that the algorithms make? Would we just print it in the screen like application logs? I'm not sure that will be viable since logs can actually get lost. Maybe write on local files as CSV or HDF5? And perhaps make a cron or something to read those files and persist it into somewhere else? Or is there a solution that runs in memory but persists to disk?

pnako · Accepted Answer

The hot path involves the network card, the CPU, RAM and nothing else. The trading algorithm will keep various data structures in memory to keep track of current position and position limits.
Outside of the hot path/thread, you can do pretty much whatever you want. You typically broadcast your trades on the network for other systems to pick them (and put that data into SQL databases, typically). The databases used for risk and compliance, etc. will most likely get their data directly from the exchange.
The idea is that you don't need to query a central system for every single trade you want to make. HFT systems have some limits set up at the beginning of the trading session, telling they can take a position of up to X in security Y.