[ts-gen] An example of multiplexing downstream using popen3() and threads

[Ken Feng]

Following Bill's excellent suggestions here:

http://www.trading-shim.org/pipermail/ts-general/2009-August/000570.html

I decided it was time to try this out - unfortunately, it's the
weekend already, so I wrote a fakey mihs.rb script which sits in place
of the shim - all it does is reverse the characters coming in from
STDIN and sends them to STDOUT:

------------------------------------------------------------------------
mihs.rb
------------------------------------------------------------------------
#!/usr/bin/ruby

while (line=STDIN.gets)
  sleep 3
  puts line.chomp.reverse
  STDOUT.flush
end
------------------------------------------------------------------------

Now, in Bill's post, he mentions using popen3() and select() in Ruby.
I couldn't find any good documentation on select() so I thought I'd
try popen3() with threads instead.  In this simple mihs_client.rb, I
have two threads:

1. Listens to mihs.rb for responses and dumps them to STDOUT - also
echos the result back to mihs.rb, just to make sure both threads can
write to mihs.rb
2. Listens to the database side and writes out to mihs.rb - I wrote a
fakey stub here that just prints foobar incremented by one each time.

NOTES:
1. Expect both files to be in the same directory with chmod +x
2. You can run the script in simple mode (without threads for
testing/understanding) or with threads - just check the last few lines
in the file.
3. You obviously need to be careful when writing to IO from both
threads - use a Mutex like in the example below is one option.  Since
I only have one thread listening, don't need a Mutex there.

Threading works okay for me since I can poll my database for
orders/signals at regular intervals and get away with it.  I suspect
using select() should result in more elegant code.  If I remember
right, select() should allow for asynchronous reads from the database
(order/signals) side.

If someone does get select() code working in Ruby, it would be nice if
you can share your results.

Nonetheless, I hope someone finds this post useful.

- Ken

------------------------------------------------------------------------
mihs_client.rb
------------------------------------------------------------------------
#!/usr/bin/ruby

require "open3"
require "thread"

class Mihs
  def initialize
    @stdin, at stdout, at stderr = Open3.popen3( "./mihs.rb" ); sleep 1
    @puts_lock = Mutex.new
    puts_threadsafe( "Hello World" )
    puts_threadsafe( "GoodBye" )
  end

  def puts_threadsafe( s )
    @puts_lock.synchronize {
      @stdin.puts s
    }
  end

  def poll_from_mihs
    while line = @stdout.gets do
      puts "RESPONSE = '" + line.chomp + "'"
      puts_threadsafe line
    end
  end

  def poll_from_db
    cnt = 1
    while true
      sleep 1
      puts_threadsafe "foobar" + cnt.to_s
      cnt = cnt + 1
    end
  end

  def startThreads
    threads = []
    threads << Thread.new { poll_from_mihs }
    threads << Thread.new { poll_from_db   }
    threads.each { |aThread| aThread.join }
    puts "mihs exiting"
  end
end

mihs = Mihs.new
simple_test = true     # turn this to false if you want to try threads
if ( simple_test )
  mihs.poll_from_mihs
else
  mihs.startThreads
end
------------------------------------------------------------------------