[ts-gen] Examples, and mysql, and ssl

Tue Jan 13 21:29:36 EST 2009

On Tue, 13 Jan 2009 15:34:54 -0500, "Bill Pippin" <pippin at owlriver.net>
said:
> Justin,
> 
> With respect to the manual:
> 
> > So far I seem to be able to connect to the api, and access the demo
> > account, I'm now at the "Using the Shim" section in the manual.  
> 
> You should be aware that the manual has become increasingly out-of-date
> over the last year.  The shim is under active development, and we are
> adding features on a day to day basis at this point, which features
> include new sentences in the command grammar.  The example scripts in
> the exs directory plus the mailing list are your best guides at this
> point.
>

I'm going to mention as much as I can through my trials and errors with
the shim to 
aid in the development of the program and documentation. I'd like to
clear something
up with me becoming sidetracked with the ACID feature and the
serializable mode. 
If I would have read on to the next page of the manual it would have
saved me the 
past problem in regards to switching it over to ansi mode. The
instruction in getting 
the whole program up and running is actually well thought out.  

> 
> I'm glad to see you posting to the mailing list; in my experience,
> once most people get started, they find the shim pretty easy to use,
> and the people reading the list should be able to help you over the
> rough spots.  The big obstacle seems to be the initial installation,
> setup, and configuration, which you've already achieved.
> 
> > I'm still contemplating on whether I should invest more time
> > familiarizing myself with mysql ...
> 
> You don't need to have any special skills with mysql.  The configuration
> of the sql isolation level is an issue with nearly all sql vendors; they
> set it too low, to get artificially high benchmark performance, and sane
> dbas reset it to serialize reads. The mysql conf file change is a one
> time problem, documented --- correctly --- in the manual, and you've
> already taken care of it.
> 
> The key skill you need with mysql is starting up the interpreter,
> which they call mysql.  Once started, it's equivalent to the isql
> (Interpreted-SQL) translator that most vendors ship.  Type or pipe in
> a query, and you get a result, either to the screen, or the stdout.
> All you need to use the mysql interpreter is an understanding of the
> SQL language, in particular the select statement, and there are plenty
> of examples in the sql directory.
> 
> As for other database understanding, ideally, you might want to be able
> to read sql create table statements, and understand the implications of
> primary and unique key declarations, and be able to view and understand
> the foreign key dependency graph diagrams we ship with the shim.  This
> level of understanding is not essential, but it's helpful, especially
> if you want to subscribe to unusual securities, or track the performance
> of your trading strategy using the order journal.
> 
> With respect to ssl:
> 
> > ... ssl disabled ... I get:
> 
> >           The trading shim has connected to the database server
> >           and loaded 51649 products; connected to the IB tws as
> >           client id  1; and queried for the account code, which
> >           seems to be DU15082.  Program initialization has been
> >           completed.
> 
> Good.  Congratulations, you've got the shim started with successful
> connections to the database and tws.  You should be able to use
> the shim, say with example scripts from the directory exs, which is
> parallel to, that is in the same unpack directory, e.g., shim-090112,
> as src, lib, and sql.
> 
> Now, about the timing problems you are seeing:
> 
> > I tried connecting to the tws with ssl enabled ...
> 
> > 	Problem: 421 the client id  1 is ignored by the IB tws
> > 	Problem: 421 the client id  2 is ignored by the IB tws
> > 	Problem: 421 the client id  3 is ignored by the IB tws
> >       ... 
> > 	Problem: 524 could not obtain IB account info at startup
> 
> The 421 messages might in other circumstances indicate that multiple
> shims were connecting to the IB tws.  Here, however, for the first
> shim started, the exception error messages indicate timeouts, at
> least four of probably two seconds each.  These timeouts can only
> occur at startup, interspersed with or instead of the banner; once
> it has displayed, you're home free.
> 
> The shim --- and, for that matter, the tws api --- is specified in
> terms of a tcp socket connection.  Once it has finished the upstream
> connection handshake, the shim uses nonblocking reads, happily
> accepting input from either the downstream client scripts or the IB
> tws whenever such input arrives.
> 
> That being said, we don't want our downstream scripts to hang waiting
> on the shim if something goes wrong at startup, so we haven't written
> the shim to poll forever during the handshake; instead, it waits at
> most some limited time, and then times out.
> 
> This time limit is controlled by the shim configuration variable
> PollTime, which may be used in a .shimrc configuration file.  The
> sample .shimrc file that is shipped with the sources has up 'til
> now had an initial value of two seconds, which I've now increased
> to three seconds, in part because of your report.  You can set it to
> whatever suits your purposes; just change the right-hand side value
> for the entry in the .shimrc configuration file, found in the
> directory where you unpacked the sources.
> 
> So, feel free to increase the value of the PollTime variable in
> the .shimrc file, although you may want to read furthur, and consider
> carefully whether there is some other issue you want to deal with as
> well.
> 
> First off, have you marked the IP address the shim is connecting
> from as a trusted address in the IB tws?  If you are manually clicking
> on a dialog box to accept the connection, you should expect the shim
> to timeout.  I realize this is an unlikely problem, but it's worth
> checking before we go furthur, if only because it is so easy to fix.
> 
> Second, you shouldn't put too much weight on what you see with
> the demo account; it's very flaky.  Paper accounts work pretty well,
> but the demo accounts can show any number of odd behaviors.  You do
> have an IB account, right?  Please be aware that without one, you
> can't collect history, or get meaningful price data, or trust the
> executions you see.
>

I thought that the demo account was the paper account, I had done a
search on the
sight and didn't realize it had to be set up through the account
management.  I just
finished setting it up and will be able to use it by tomorrow or at the
latest by
thursday. 

The localhost IP is set in the Configure/API section of the TWS, it auto
accepted with
ssl disabled. 

> 
> > Should I be able to connect with ssl enabled?
> 
> Third, please realize that the delays you are seeing are due not to
> the shim, but rather the IB tws; the choice of ssl determines how it
> connects to its upstream.  The tws api, and the shim's connection via
> that api, uses a regular, unencrypted socket.
> 
> So, the IB tws is connecting to its upstream over a network layer
> that adds multi-second time delays.  If those are only at startup,
> e.g., for certificate lookups, then the delays may well be harmless,
> and you could just wait to start the shim until the IB tws has
> finished talking to its servers.  Beyond this, your choices are
> simple: either turn off ssl; or increase the time until the shim
> would give up waiting.  For the latter, as explained above, use the
> shim's timeout variable PollTime to change the maximum wait for the
> connection handshake and account query; you can increase it as desired.
> Otherwise your problem is outside the control of the shim; if the IB
> tws wants to take its time talking to the upstream, there's not much
> else the shim can do but wait.
>

I just realized that the demo account might have a  restriction on ssl
use, who needs
ssl on a demo account.

Russ mentions using the The IBGateway, the only problem with this is
I'll need a 
separate account for the CTCI order management, as well as another to
receive data 
through the API. Not only that they require a minimum of 100 USD monthly
in 
commisions to use this. 

The explination listing the trading-shim under the freelance section of
IB says,

"The command interface also serves as an alternative to the existing IB
TWS 
GUI interfaces, to permit you to use additional downstream programs as
well, 
whether GUI or not, to drive the IB TWS through its API."

There are two things that I was sold on, command line & alternative to
the existing
IB TWS GUI interface. 

I'm also interested to know if anyone has used the data stored in mysql
with another
charting program.

Thanks for your feedback, and direction.

Justin

> Thanks,
> 
> Bill
> 
-- 
  Justin
  justins at ftml.net

-- 
http://www.fastmail.fm - Access your email from home and the web