[ts-gen] OrderStatus dups and skips

[Bill Pippin]

Nils,

In brief, I'd like you to consider trying out today's release of
the shim.  You should find that duplicate OrderStatus records do
not occur, and I very much want to know whether OrderStatus inserts
are lost/skipped, e.g., the missing "Submitted" records you mentioned,
once given that the related log message *was* received by the shim.

In more detail, about the duplicate OrderStatus lines, which I've
observed:

> shim-090713 works fine, no segmentation faults. For some reason I
> am oserving duplicate OrderStatus lines. 

and your earlier report of intermittent lost OrderStatus records,
which I have not:

> I observered incomplete updates especially of the table
> OrderStatus; for example  'Submitted' of an record missing, then
> 'Filled' coming in.

To recap context for other readers:

    1.  The IB tws frequently sends duplicate order status messages
    2.  The shim has followed a policy of posting order status,
        open order, portfolio, and execution reports to the journal
        nearly one for one, with the qualification that dups should
        be ignored for order status, open order, and exec reports.
    3.  Orders, and order status messages, are by their nature
        per-account, and yet unfortunately IB tws api order status
        messages do not include the account code.
    4.  Until recently the shim followed a policy of adding the
        account code to such records by using the account code that
        is obtained --- and displayed in the banner --- at startup.

Here's how the problem with dup inserts occurred:

    5.  Unfortunately, there is a possible race in such processing;
        the IB tws can be configured to send status updates for
        certain order types, and it might send such *before* it
        replies to the account data query the shim uses to determine
        the account code, in which case table lookup, and so record
        creation, would fail for those order status messages read
        at this early stage of program initialization, before the
        banner is finished.
    6.  In response, I recently relaxed the sql declaration for 
        the account code attribute of the OrderStatus table to accept
        nulls, and revised the insert processing to leave out the
        account code, since it might not be available.
    7.  In a related change, I also commented out several unique key
        declarations in OrderStatus, since without the account code
        uniqueness was lost.
    8.  Although there was also code in the shim to check for
        duplicates, it is problematic, about which more later.
    9.  In any case, once the shim's check for duplicates failed,
        the lack of uniqueness declarations in the OrderStatus
        database table declaration allowed duplicate inserts to
        occur.

As far as the database goes, I've now restored account code foreign
key checking, the not null declaration for acc attribute, and unique
key declarations for the OrderStatus table, so that duplicate inserts
no longer occur.  The database version has been incremented, so you'll
need to recreate your database.

This leaves your earlier problem of intermittent skips for
OrderStatus inserts.  Recall that I mentioned in (8) above that
the shim's duplicate checks fail.  I've commented out this code
for now, since broken code may well fail in multiple ways, and
in any case the uniqueness key checks by the database are by
themselves sufficient to prevent dup inserts, although at some
cost in efficiency.

Once you've been running this shim long enough for you to have
expected to observe missing inserts, I'd like to know whether in fact
you see any.  After all, for an intermittent problem I haven't yet
seen for myself, I don't know for sure at this point whether it's
fixed.

Your report will help me narrow down remaining problems in this
area, if any; I'll then be able to go on and write code to block
order status updates prior to account data initialization, to rule
out the race condition mentioned above, and fix the shim's dup
checking code.  As it stands now, I don't want to make multiple
changes in this functional area before you have a chance to test
to see if the intermittent problem you observed earlier still
occurs.

Thanks,

Bill