A Database System

The api wire formats imply elaborate contract and order objects, and the permanent definitions for these may be conveniently stored in a database, and referred to by name. This notion of persistent contract and order records is the key to simplifying the command language used by the downstream to make requests. Since for the number of symbols of interest, persistent data can be obtained from the database all at once, there need be no time cost for contract and order database access once past initialization, and in fact the shim has been implemented to slurp the entire database into memory at startup.

Once given the fundamental purpose of supporting shared definitions of contracts and orders between downstream clients and the shim, the database provides in addition a store for historical data, a journal for orders, and a means to dynamically update the shim during process execution. Though such access does cost time, it is either relatively infrequent, as in the case of order journal entries, or can be offloaded to a subprocess, as when history data is saved.

At this time history query writes are still performed by the main process. Although this is not a problem for small, recent queries, e.g., the last hour's worth of 15-second data, it is probably best for the time being to use distinct process instances of the shim for large history queries and orders, respectively. Since, one, there should be no need to perform large history queries as part of a high-frequency trading strategy, and two, up to four instances of the shim can share the same IB tws server, as it accepts up to eight api socket connections, and the shim uses at most two, then it is perfectly feasible to partition bulk data collection and order computation in this way.