[ts-gen] Shim config variables [Was: multiple shims, client id's]

[Bill Pippin]

Ken,

About the fix for the problem with concurrent sessions reported by
PaulC:

> Thank you for implementing this.

You're welcome.  And now about your question relating to the ClientId
variable:

> My only question is, the ClientID key-value pair in .shimrc
> appears to be neglected.  When I change it to ... ClientId 3
> [and] ... I try to bring up a new ./shim --data ...

The ClientId variable is ignored in --data mode, and data mode
connections are clamped to use ids starting at 1 and at most 8.

By convention, ClientId numbering for --risk mode connections
starts at 8, and users should think carefully before trying to
run eight data mode connections, since existing positions for
that account become inaccessible via the api not only until you
terminate the last data mode shim, but also until the IB tws
releases that connection id.

> Now, I actually prefer your implementation where you increment
> client id internally and post a warning [422 message] ...

> Problem: 422 the client id  1 has not yet been released

You'll only see these with multiple data mode sessions.  By the way,
I'm reposting below most of a prior message from last year that gives
some explanation of the available configuration variables.  I've edited
it lightly, and added an explanation of the ApiLevel variable, which is
new, near the end.  Repost, subj: .shimrc file, dated Fri, 22 Aug 2008:

________________________________________________________________________

The release script has been modified to again provide a
useful default .shimrc file in the root directory of the
tarball:

DbmsName        mysql
DbmsHost        localhost
TableSet        testing
UserName        shim
Password        0
FeedName        tws
FeedHost        localhost
FeedPort        7496
HqPeriod        11
ClientId         8
Upstream        paper

All variables have reasonable defaults, so that it's fine
to minimize this file to the connection values that vary
for your site, plus possibly your preferred value for
ClientId. 

The set of possible name value pairs was previously available
via help, though since that is currently disabled, the
determined user will have to look at the text of help.c to
see that information.  [the help text here has become increasingly
out of date --- note added 090825] ...

The language accepted while parsing rc files is determined by ...

(2) the declarations of the pair names, in kind.h, which indicate
whether the pair value is a string or natural number, as the 
type template parent type is a [String] or [Number]; and,

(3) the definition of the set of pair names acceptable in actual use,
by the list of names in the table returned by tag::RcpK::instances()
of syms.c, where the tag type acronym stands for "rc-pair-keyed set",
and the current list from that file is:

  a.  DbmsName DbmsHost TableSet UserName Password
  b.  FeedName FeedHost FeedPort
  c.  ShimText CmdEvent ReqEvent MsgEvent
  d.  Locality Currency Account Upstream

  e.  ...      ClientId
  f.  NumTries DataSubs BookSubs HqPeriod
  g.  ...      PollTime NumWords InitTime

The first five, in (a), are dbms and database connection parameters;
those in (b), IB tws connection parameters; (c), log and trace file
names; (d), upstream session and account context --- all of these are
strings, although I should note that strings may be numbers, e.g., a
database password value of 0 indicates the empty password, and the
FeedPort may be either a port number, e.g., 7496, or for more exotic
setups, a service name.  The variables in (a), (b), and (c) have
been around for quite awhile, and are fully implemented as intended.

The values in (e) also provide upstream session ... context;
those in (f), resource limits; and (g), somewhat obscure internal
timeout delays, counts, and other configuration values used only
during startup, so that none of this last group are of interest
once the shim has finished printing its banner.  [Most of the
variables in this last category, g, are more or less obsolete;
you should probably leave these alone unless you know what you're
doing; note added 090825].

I'll leave (a) and (b) to your common sense, accumulated experience,
or, as needed, perusal of help.c .  The defaults are typically fine
for the log file names, (c), and timeouts, (g).

The resource limits (f) are described in a NEWS stanza within the
current series:

>   The NumTries, DataSubs, and BookSubs configuration variables
>   have been added to control the number of connection attempts,
>   number of market data subscriptions, and number of market
>   depth subscriptions per shim, e.g., with a DataSubs value of 12,
>   eight shims sharing the same config file could make up to 96
>   market data subscriptions.  The "once" cmd line option flag
>   overrides the NumTries variable, so that say in risk mode,
>   and where you want to have exact control over the ClientId,
>   the shim will quit after the first connection attempt if
>   there is a client id collision.  The HqPeriod variable,
>   renamed from PastSlot, can be used to divide up the history
>   query bandwidth; e.g., with a HqPeriod of 81 seconds,
>   eight shims should be able to comfortably share one IB tws.

That leaves the upstream session context, (d) and (e):

  d.  Locality Currency Account Upstream
  e.  ...      ClientId

Although most of these are also covered in NEWS, I'll reiterate here
since [the older text in that] file does by its nature become outdated.

The ClientId variable, which applies only to risk mode [data mode shims
are clamped to the lower single digits, from 1 to at most 8], is
perhaps the most interesting of the newer configuration variables.
Use of the client id and region variables, (names "ClientId"
and "Locality") are explained in a previous post:

    http://www.trading-shim.org/pipermail/ts-general/2008-August/000228.html

... Account and Upstream are currently ignored, that is not used during
configuration.  ...

[Currency and Locality now have significant effects, and should be
changed if the defaults do not match your area and account setup,
since symbolic product expressions have now been added to the command
language; note added, 090825]

... Account and Upstream would allow you to limit which IB tws instances
the shim could connect to, according to the account code or class (paper,
live, demo, or paper/demo).  [Other than as noted, the variables in (d)
are still not completely implemented, and their planned role is subject
to change; note added 090825.]

The client version variable,  cVersion ... [has been eliminated; read
on for the added explanation of ApiLevel:]

________________________________________________________________________

The older config variables sVersion and cVersion have been eliminated.
There is a new config variable, ApiLevel, which can range from 23
to 44, although you should leave it at 23 unless you know what you're
doing, most likely because you've read the NEWS where I explain now to
get preliminary access to newer api features, and you understand
that full api support past 23 is not yet implemented.

E.g., you can increase the number of retrieved attributes in contract
data messages by using an appropriately higher api level, but other
features will not work at that level, so it's just for contract
data collection.

The ApiLevel variable will eventually allow you to select api request
and message formats to the extent they're supported by your IB tws
client.

Thanks,

Bill