[ts-gen] Trouble with new syntax for stocks

[Bill Pippin]

Paulo,

Thanks for trying out the new shim:

> ... installed the most recent shim version here (Ubuntu 8.04)
> yesterday to give the new syntax a try for retrieving historical
> stock data.

Although I suspect our regression test examples, e.g., exs/tick ,
would have worked fine for you, it's clear that European symbols
were not being accepted:

> ... the new [syntax] only leads to "Syntax error" messages. ...
> ... following lines ... [from stderr] ... to dump ... data of
> german stock Metro (MEO) routed at Xetra (IBIS).

Thanks for the excellent diagnostic info:

> select past STK:IBIS:MEO:EUR m1   1  1d Ymd_T(20090316  18:00:00);    

> Process rule: NonInput (ValA)
> Process rule: Compound (STK)
> Syntax error:
> Grammar rule: Compound STK    
> Cursor state: match; text is:
> select past STK:IBIS:MEO:EUR m1   1  1d Ymd_T(20090316  18:00:00);
> ----------------------------^-------------------------------------

> I wonder what went wrong here.

The short answer is probably that you need to set the Locality to
the correct value in your .shimrc file; though please read on.

In the first trace line from the parser, a grammar action has failed.
Action names typically end in A, and in any case are labelled as
"NonInput", since they are neither a terminal nor nonterminal grammar
symbol.  Then, given that the table lookup of the symbol expression
value has failed, the overall grammar rule match fails as well.
Another clue is that the error point caret follows the expression
text; each token was accepted just fine, and after that, something
went wrong.

So at this point, we know the shim is unable to find the symbol
expression STK:IBIS:MEO:EUR in its internal database.

Now, a key point to keep in mind; the attribute value pairs in
the .shimrc file for Currency and Locality should be correct,
and the Locality pair *must* be there for non US symbols; Currency
is optional, since it's explicit in the contract value expression.  

Since the default ships with USD and US, you'll need to change these
before working with your preferred symbols.  So, in what follows, it
may help to know that I added lines for EUR (correct) and EU (in
this case, incorrect) to my .shimrc file to try to duplicate your
problem, and when I tried your example, or for that matter any
other STK:IBIS:MEO:EUR subscription, I saw exactly the same problem.

A contract info query for STK:IBIS:MEO:EUR worked just fine; given
the command:

    select info STK:IBIS :MEO:EUR;

The following (trimmed) contract data message showed up in the log:

    3|10| 5|MEO|STK||0.0||IBIS|EUR|MEO|XETRA|XETRA|1907293|...

Searching for 1907293 in the database symbol listing:

    sql$ grep 1907293 syms.txt

extracted the following text:

    14756   1907293 STK     MEO     EUR     DE

This indicates the answer; the locality has to be DE .

I had earlier uncommented the following line in the symbol value
search loop, in leaf.c, part of Tables::operator[]() :

//      d.print_line(t[i], a, b, x);

and this gave me traces of the previously failed table lookups.
Once I fixed the Locality to DE in the .shimrc, I got the following
trace with success at the end:

Pidx: <     1 25826 STK  STK, EUR JPY,  DE  JP, MEO !=    6274: 14016454
Pidx: > 12914 25826 STK  STK, EUR USD,  DE  US, MEO !=     UVU: 43137502
Pidx: < 12914 19370 STK  STK, EUR CAD,  DE  CA, MEO !=  BTB.UN: 41140714
Pidx: < 12914 16142 STK  STK, EUR EUR,  DE  GB, MEO !=     AXS: 44081627
Pidx: > 14529 16142 STK  STK, EUR EUR,  DE  DE, MEO !=     FTM:  4810995
Pidx: < 14529 15335 STK  STK, EUR EUR,  DE  FR, MEO !=   ALCLA: 43630950
Pidx: < 14529 14932 STK  STK, EUR EUR,  DE  DE, MEO !=     RWD: 13891882
Pidx: > 14731 14932 STK  STK, EUR EUR,  DE  DE, MEO !=     LSX: 13640855
Pidx: < 14731 14831 STK  STK, EUR EUR,  DE  DE, MEO !=     O4B: 40345329
Pidx: < 14731 14781 STK  STK, EUR EUR,  DE  DE, MEO !=     MPS:  4155574
Pidx: = 14731 14781 STK  STK, EUR EUR,  DE  DE, MEO ==     MEO:  1907293

As for my attempts to subscribe for the symbol, I received errors,
but I believe those are due to my account setup, and outside the
control of the shim:

3| 4| 2|2|\
 354|Requested market data is not subscribed. Error&IBIS/STK/Top|

So, if you fix the config file Locality, I suspect the new syntax
will work fine for the example you give; and in general, given
problems with value-based symbols lookup, you can follow the steps
above; try contract info, check the database symbols listing, and,
if you're comfortable with source code, feel free to turn on the
trace info given by the print_line() procedure from the Tables op[]
search loop in leaf.c

Thanks,

Bill