[ts-ann] trading-shim 061204 announced
R P Herrold
herrold at owlriver.com
Mon Dec 4 21:27:33 EST 2006
Mea culpa:
I missed posting an announcement of the mid November release.
As there was much traffic in the ts-general mailing list, I
overlooked it. I just kicked a release notice into the
Freshmeat submission queue, but this piece will beat the
nightly mailer they send out. My change summary is based upon
the top of:
http://www.trading-shim.org/capitals/?NEWS
and so at any time you may look there to review recent
announcement 'fodder.'
Summary of Changes:
* build the Symbol (previously named Definition) table
from the tables Underlying and ProductMap, and Underlying from
distinct tables Stock, Currency, and Miscellany
* use symbolic, or key value, literals in place of numeric
contract literals in defining the LocalSet table
* eliminate the Security table from the dbms; there is now
a direct foreign key chain from Underlying to Symbol to
Contract
* add some non-us home symbols to Stock.sql,
Miscellany.sql, and ProductMap.sql as examples of how these
tables can be modified for the use of those outside the US
The last month's feedback has put us well along on the road to
supporting the UK and APAC users (and the desired Underlyings)
of the shim, each of whom are in a Debian 'testing' build
environment. We have become comfortable with the update
process, and have the code performing under the gcc-c++, ver.
4 compiler which it uses.
Most that were requested are now present; order management,
market data subscription, and history retrieval for the
IdealPro Forex 'exchange' operated by IB are wholly untested
by us, and we are particularly interested in reports in that
aspect. Stocks, Index, and Futures all seem to work fine in
our testing.
A couple of the requested tradables are not available to us
[As we were re-examining the database's schema, we had not yet
formally opened a testing paper account with those rights, and
need to do so, but other priorities have been at the fore,
with the US Thanksgiving holiday], and so SGX (which seems to
be an Exchange, an Index, and a Stock), and the TSB, are not
in today's external release.
We have re-worked the sql database load scripts to reflect the
schema changes outlined above, and as a result, the dataset
version is bumped to: dbms version 1.17
On the Debian box, we do a quick self-test, and inventory of
the libraries linked; then a long --help list of command modes
and options. This is the slowest of our boxes, having no
useful distccd (it has no peers of a proper g++ level to
connect to), and being a P-II 333MHz, with a single 'old
clunker' IDE drive and 256 M of ram
herrold at 0[shim-061204]$ echo "quit;" | ./shim --cmds
quit;
herrold at 0[shim-061204]$ ldd shim
libmysqlclient.so.14 => /usr/lib/libmysqlclient.so.14 (0x40035000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x40150000)
libm.so.6 => /lib/libm.so.6 (0x40235000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x4025a000)
libc.so.6 => /lib/libc.so.6 (0x40265000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x40380000)
libnsl.so.1 => /lib/libnsl.so.1 (0x403ae000)
libz.so.1 => /usr/lib/libz.so.1 (0x403c3000)
/lib/ld-linux.so.2 (0x40000000)
herrold at 0[shim-061204]$ ./shim --help
Usage: shim <mode> [optional feature list]
Modes:
--help # print this explanation and list the optional features
# real modes, requiring access to an IB tws:
--data # process subscriptions and log resulting tick stream events
--risk # accept full command set, send requests, and log all events
--leaf ** # shsh experimental use, subject to rapid and drastic change
# test modes, with no connection to the tws:
--unit # for internal use; unstable though otherwise harmless
# echo modes; read from stdin and echo the related:
--cmds # downstream commands
--msgs # tws messages
--reqs # tws requests
--play * # read events from the image file and send text to stdout
# allowed options may be listed following the mode:
# real modes only:
init * # prompt for the dbms and link commands during startup
load # load SubRequest at startup even without a load command
cast * # multi-cast tick, depth, history, and bulletin messages
dump * # read the database tables and list their rows to stdout
save # save events to image and text files for later playback
pipe # send events as text down a pipe to provide a local log
** # leaf mode only, and must be suffix of the option list:
part # introduce the message partition definition; each event
# type listed will be split to a separate output stream:
past # history
tick # market data and market depth
news # news bulletins
mkts # all of the above
# test modes only:
fast # cut startup time; this saves just one second, works with
# Linux only, and is used primarily to speedup the offline
# parts of the test suite
* -- not yet implemented
** -- for now, expect to read the code if you wish to use this
herrold at 0[shim-061204]$
Interesting newly supported 'real' mode modifier options
include:
save # save events to image and text files for later playback
pipe # send events as text down a pipe to provide a local log
which are designed to essentially 'capture' the datastream
from the upstream TWS into a file or pipe (a pipe under *nix
is of course a somewhat special type of file, but a file
nonetheless); this is of use in capturing every 'byte' of data
without resort to 'tcpdump'. The data then may be fed to 'od'
for a variety of visualization formats.
On the Debian box, our build process looks like the following
extract. Please note that even internally, (as noted in the
INSTALL file, section 2, _we_ patch in the TWS and database
changes we need (along with doing a bit of self testing that
the database is alive and reachible). I choose to to automate
the testing copy with a cleanly installed database at the
start of each major session as well.
http://www.trading-shim.org/capitals/?INSTALL
herrold at 0[shim]$ time ./patch-script.sh shim-061204.tgz
+-------------------------------------------------------------+
Info: patch-script.sh version 0.14-061204 called thus:
./patch-script.sh shim-061204.tgz
Warning: do not see any: ~/.shim-patchrc
+-------------------------------------------------------------+
Info: database details: host ftp.first.lan, dataset doco_trender
by userid shim with a password
Info: scratch reload of database starting
Info: scratch reload of database done
Info: shell Database version dump
uid current text
17 current shim dbms version 1.17
Info: PHP Database connection verification
patching file once.c
Hunk #1 succeeded at 292 (offset 14 lines).
patching file data.c
patching file includes
Hunk #1 succeeded at 116 (offset 106 lines).
Info: all patches seem to have applied properly
+-------------------------------------------------------------+
distcc g++ -Wall -g -I/usr/include/mysql -c -o boot.o lib/boot.c
distcc g++ -Wall -g -I/usr/include/mysql -c -o kind.o src/kind.c
... snip ...
distcc g++ -Wall -g -I/usr/include/mysql -c -o pool.o lib/pool.c
distcc g++ -Wall -g -I/usr/include/mysql -c -o unit.o src/unit.c
date; distcc g++ -g -o shim once.o make.o init.o mode.o
else.o boot.o data.o type.o tabs.o kind.o atom.o load.o shim.o
time.o read.o flow.o send.o post.o wire.o echo.o term.o leaf.o
call.o hash.o pool.o unit.o -L/usr/lib/mysql -lmysqlclient -lm
Tue Dec 5 04:37:20 CET 2006
... snip ...
real 19m57.537s
user 16m14.780s
sys 1m53.310s
herrold at 0[shim]$
The release is ready for your perusal and testing at:
ftp://ftp.trading-shim.org/pub/local/trading-shim/shim-061204.tgz
Thank you for your interest; we welcome feedback and comment.
-- The trading-shim team
More information about the ts-announce
mailing list