Hi Bill,<br><br>It turns out that there is a documented bug in mysql versions 5.1+ which causes the gap in the symbol tables -- the gap occurs when trying to place many rows in the table simultaneously.<br><br><a href="http://bugs.mysql.com/bug.php?id=40384&error=lp">http://bugs.mysql.com/bug.php?id=40384&error=lp</a><br>
<br>Using version "14.12 Distrib 5.0.77" solves the problem.<br><br><br>Thanks.<br>Mike<br><br><br><br><div class="gmail_quote">On Wed, Apr 15, 2009 at 12:11 PM, Bill Pippin <span dir="ltr"><<a href="mailto:pippin@owlriver.net">pippin@owlriver.net</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Mike,<br>
<br>
Thanks much for your report. You've already run into teething<br>
problems due to my preliminary work on options support, and now<br>
you may be having knock-on problems due in part to some cruft in the<br>
setup scripts that I overlooked, that most people don't see, but<br>
that may have been uncovered by the particular sequence in which<br>
you've been doing the setup. Lastly, you will in addition see a need<br>
for some more entries for FUT:ECBOT:YM symbols before the regression<br>
tests will work properly, and for these you'll need to either apply the<br>
patch I include below, or else download the newest data-090415 tarball<br>
in order to add the symbols. I very much appreciate your help in<br>
identifying these problems. Sorry about the headaches.<br>
<br>
> Running "mysql_install_db" again + other minor file edits ... then<br>
> sourcing "setup.sql" cleaned up most of those errors. My<br>
> last error (I hope) is a gap in the Symbol table. ...<br>
<br>
I've already pushed the new symbols tarball out, and it should show up<br>
within the hour on our site.<br>
<br>
I've tried to duplicate what you're doing, and found a problem.<br>
That being said, I would expect the setup.sql script to run for you,<br>
and suspect that you may also be having some permissions issues.<br>
<br>
The setup script issues I've found will be fixed in the next release of<br>
the shim sources, that is a tarball shim-0904??.tgz, which I hope to put<br>
out either today or tomorrow. Rather than ask you to wait, however, I'll<br>
tell you how to diagnose the problem if you wish, and in any case, where<br>
the script problem I see is, and you can fix that yourself.<br>
<br>
What follows is a detailed recipe for fixing all the various issues I've<br>
noticed that you might be running across. Please feel free to ask<br>
questions as needed.<br>
<br>
________________________________________________________________________<br>
<br>
A. Adding symbols<br>
<br>
Carrying out this step first will allow you to avoid repeating the next<br>
steps over again. The only dependency between this step and the other<br>
two is that you need a few additional symbols to allow the shim's<br>
regression tests to work properly; those symbols must be included<br>
in the database; and the following two steps do just that. So, either<br>
you want to get the new symbols before you create the databases; or<br>
else you'll probably want to repopulate those databases later.<br>
<br>
You may either apply the patch that follows, or else look it over and<br>
add the relevant entries by hand, or else download the data-090415<br>
symbols tarball. In the latter case, you'll need to either copy the<br>
file data-090415/A0Fut01.sql into data-090320, or else update the<br>
top symlink in the shim source directory, the one that points up and<br>
over, to point to the new symbols directory.<br>
<br>
diff -rc data-090320/A0Fut01.sql data-090415/A0Fut01.sql<br>
*** data-090320/A0Fut01.sql Wed Jan 21 12:25:34 2009<br>
--- data-090415/A0Fut01.sql Wed Apr 15 14:07:26 2009<br>
***************<br>
*** 2340,2345 ****<br>
--- 2340,2349 ----<br>
(49652399, 'FUT', 'DD' , 'USD', 'US' , '20090319', 'DD MAR 09' , 'BIG SIZED DOW JONES INDUSTRIAL AVERAGE $25' , '' , 'ECBOT' ),<br>
(49652401, 'FUT', 'DJUSRE' , 'USD', 'US' , '20090319', 'RX MAR 09' , 'DOW JONES US REAL ESTATE INDEX' , '' , 'ECBOT' ),<br>
(49652404, 'FUT', 'YM' , 'USD', 'US' , '20090320', 'YM MAR 09' , 'MINI SIZED DOW JONES INDUSTRIAL AVERAGE $5' , '' , 'ECBOT' ),<br>
+ (51865515, 'FUT', 'YM' , 'USD', 'US' , '20090609', 'YM JUN 09' , 'MINI SIZED DOW JONES INDUSTRIAL AVERAGE $5' , '' , 'ECBOT' ),<br>
+ (53893055, 'FUT', 'YM' , 'USD', 'US' , '20090918', 'YM SEP 09' , 'MINI SIZED DOW JONES INDUSTRIAL AVERAGE $5' , '' , 'ECBOT' ),<br>
+ (56578477, 'FUT', 'YM' , 'USD', 'US' , '20091218', 'YM DEC 09' , 'MINI SIZED DOW JONES INDUSTRIAL AVERAGE $5' , '' , 'ECBOT' ),<br>
+ (58700627, 'FUT', 'YM' , 'USD', 'US' , '20100319', 'YM MAR 10' , 'MINI SIZED DOW JONES INDUSTRIAL AVERAGE $5' , '' , 'ECBOT' ),<br>
(49666526, 'FUT', 'PAN' , 'NOK', 'NO' , '20080918', 'MHG8U' , 'Marine Harvest' , '' , 'OMLX' ),<br>
(49666530, 'FUT', 'NSG' , 'NOK', 'NO' , '20080918', 'NSG8U' , 'Norske Skogindustrier ASA' , '' , 'OMLX' ),<br>
(49666535, 'FUT', 'OBX' , 'NOK', 'SE' , '20080619', 'OBX8F' , 'OBX INDEX' , '' , 'OMLX' ),<br>
<br>
I realize that the patch above may not make it through your email<br>
system, in which case, feel free to add the new entries for YM by hand,<br>
or get the new symbols tarball.<br>
________________________________________________________________________<br>
<br>
B. Diagnosis<br>
<br>
You may diagnose the setup problem you've been seeing to confirm that<br>
my explanation actually applies. Or, if you wish you can just go<br>
straight ahead to (C), the fix and its execution.<br>
<br>
While in the directory shim-090327/sql , check for the existence of a<br>
script "secs.sql". It's not supposed to exist, but almost certainly<br>
is there. Check for a reference to it in the script table.sql, ditto.<br>
You may also check for the existence of the tables ComboLeg and<br>
ComboSet, neither of which should exist.<br>
<br>
E.g.: echo 'show tables;' | mysql -u code -h xps400 testing | more<br>
adapted as needed to your site will probably produce something<br>
initially like the following:<br>
<br>
Tables_in_testing<br>
AccountCode<br>
AccountEdge<br>
ActiveOrder<br>
AtomTag<br>
BarSize<br>
Bool<br>
BoxOptions<br>
ChangeOrder<br>
ComboLeg<br>
ComboSet<br>
CreateEvent<br>
Currency<br>
...<br>
<br>
The problem here is that the tables ComboSet and ComboLeg have foreign<br>
key dependencies on other tables, that they are not currently used,<br>
are not initially dropped by the create.sql script, prevent other tables<br>
from being dropped, and so that script is not able to recreate the tables<br>
in a database properly. If you've been using setup.sql only, I'm not<br>
sure how this could be your problem, but the two tables ComboSet and<br>
ComboLeg are currently cruft, and so should in any case be eliminated<br>
for now.<br>
________________________________________________________________________<br>
<br>
C. Solution<br>
<br>
Remove the reference to secs.sql in table.sql, by deleting that line;<br>
go ahead and delete the file secs.sql, if you wish, since it's just<br>
a bug magnet; and use the setup.sql script (not create.sql, you need<br>
to start from scratch here) to recreate both the testing and trading<br>
databases.<br>
<br>
E.g., adapting the following command to your site's mysql server<br>
hostname, and working within the directory sql, one down from the<br>
directory where you unpacked the shim's sources:<br>
<br>
sql$ time mysql -p -u root -h xps400 --show-warnings < setup.sql<br>
<br>
Enter password:<br>
Note (Code 1305): PROCEDURE OrderUpdate does not exist<br>
Note (Code 1051): Unknown table 'Bool'<br>
Note (Code 1305): PROCEDURE OrderUpdate does not exist<br>
Note (Code 1051): Unknown table 'Bool'<br>
Note (Code 1051): Unknown table 'TsNames'<br>
Note (Code 1051): Unknown table 'TmpTabs'<br>
Note (Code 1051): Unknown table 'TmpUser'<br>
Note (Code 1051): Unknown table 'TmpHost'<br>
Note (Code 1051): Unknown table 'TmpShim'<br>
<br>
real 1m1.894s<br>
user 0m0.324s<br>
sys 0m0.032s<br>
<br>
As noted in an earlier post, the unknown table and nonexistent procedure<br>
warnings are superfluous, and should be ignored.<br>
<br>
To verify that you've reached a clean fix point, and that ComboSet and<br>
ComboLeg are gone, now try to recreate the tables in just one of the<br>
databases, and this time not starting from scratch:<br>
<br>
sql$ time mysql -u code -h xps400 testing --show-warnings < create.sql<br>
<br>
real 0m28.655s<br>
user 0m0.172s<br>
sys 0m0.036s<br>
<br>
The absence of warnings shows that you are able to freely recreate the<br>
tables, without going to the trouble of recreating the databases.<br>
<br>
At this point the shim should be able to connect to the database and<br>
read in its tables. You may also want to update your symbols listing,<br>
the file syms.txt, as follows (you may do this whenever you wish; the<br>
resulting file is FYI).<br>
<br>
sql$ time mysql -u code -h xps400 testing < bin/get_id.sql > syms.txt<br>
<br>
Please let us know if this helps.<br>
<br>
Thanks,<br>
<font color="#888888"><br>
Bill<br>
<br>
</font></blockquote></div><br>