Friday, October 12, 2007

Swig-C-Ruby-Pointers Hello World

I'm trying to write C libraries to do some things on the Mac that are not easy to do with Ruby. I found that I had to read several documents and experiment around a bit with SWIG to call a C library function that took a pointer as an argument. This is just one of those things that should not take that much time to figure out.

C library functions that take pointers as arguments and return ints (that covers most of the c library functions I have written) ultimately return an array to Ruby. Uncovering that little factoid, and the swig interface format required to make that process work, was the tough bit.

So, I'm taking a few more minutes to make sure you don't have to do that (and to make sure I don't have to do that long after I forget about this exercise).

Step Zero: get swig.

I install all open source software on the mac with fink. Fink Swig

Step One: Copy my example code - inline below.

I provide what you need to make a ruby callable module called Example that contains the C function SimpleFunc() that you will call with a little ruby test program called testsum.rb.
user-64-9-234-209:~/src/c/simplelib richbodo$ ls -l
total 48
-rw-r--r-- 1 richbodo richbodo 61 Oct 12 16:24 example.i
-rw-r--r-- 1 richbodo richbodo 42 Oct 12 15:52 extconf.rb
-rwxr-xr-x 1 richbodo richbodo 90 Oct 12 16:41 get_it_done.sh
-rw-r--r-- 1 richbodo richbodo 46 Oct 12 16:36 sum.c
-rw-r--r-- 1 richbodo richbodo 24 Oct 12 15:46 sum.h
-rw-r--r-- 1 richbodo richbodo 67 Oct 12 15:57 testsum.rb
Three of these files you already understand:

sum.c - your library
int SimpleFunc(float *modify_me)
{
*modify_me = 5;
return 1;
}
sum.h - your library header - not used by swig unless you tell it to - don't need it here.

testsum.rb - ruby program that calls your library
require 'example'
afloat = 2
# The first return value gets set as expected in first_returnval.
# However, SWIG puts the float in a second return value.
first_returnval,second_returnval = Example.SimpleFunc(afloat)
puts "Your float is #{first_returnval}, and the return value is: #{second_returnval}"
The other three files are specific to getting your swig library going:
  • example.i - a definition of your library for swig in swigs definition language. INOUT is a SWIG typemap, so typemaps.i is required.
%module example
%include "typemaps.i"
%apply float *INOUT { float *modify_me };
int SimpleFunc(float *modify_me);
  • extconf.rb - a ruby program that creates a makefile for your library
require 'mkmf'
create_makefile('example')
  • get_it_done.sh - the commands that you run to generate a ruby module from your c library using swig
#!/bin/bash
rm *.o
rm *.bundle
rm wrap_example.c
rm Makefile
swig -ruby example.i
ruby extconf.rb
make
sudo make install
ruby testsum.rb
Step 2: Run the get_it_done.sh script to build and test your module. (it will ask for your password to install the library)
user-64-9-234-209:~/src/c/idlerlib richbodo$ ./get_it_done.sh
rm: wrap_example.c: No such file or directory
creating Makefile
gcc -fno-common -g -O2 -fno-common -pipe -fno-common -I. -I/sw/lib/ruby/1.8/i686-darwin -I/sw/lib/ruby/1.8/i686-darwin -I. -I/sw/include -c example_wrap.c
gcc -fno-common -g -O2 -fno-common -pipe -fno-common -I. -I/sw/lib/ruby/1.8/i686-darwin -I/sw/lib/ruby/1.8/i686-darwin -I. -I/sw/include -c sum.c
cc -dynamic -bundle -L"/sw/lib" -o example.bundle example_wrap.o sum.o -lruby -ldl -lobjc
install -c -p -m 0755 example.bundle /sw/lib/ruby/site_ruby/1.8/i686-darwin
Your float is 5.0, and the return value is: 1
Step 3: Look at the five new files you just created, and understand what they do:
user-64-9-234-209:~/src/c/simplelib richbodo$ ls -l
total 152
-rw-r--r-- 1 richbodo richbodo 3009 Oct 12 16:42 Makefile
-rwxr-xr-x 1 richbodo richbodo 16316 Oct 12 16:42 example.bundle
-rw-r--r-- 1 richbodo richbodo 61 Oct 12 16:24 example.i
-rw-r--r-- 1 richbodo richbodo 17695 Oct 12 16:42 example_wrap.c
-rw-r--r-- 1 richbodo richbodo 5744 Oct 12 16:42 example_wrap.o
-rw-r--r-- 1 richbodo richbodo 42 Oct 12 15:52 extconf.rb
-rwxr-xr-x 1 richbodo richbodo 90 Oct 12 16:41 get_it_done.sh
-rw-r--r-- 1 richbodo richbodo 46 Oct 12 16:36 sum.c
-rw-r--r-- 1 richbodo richbodo 24 Oct 12 15:46 sum.h
-rw-r--r-- 1 richbodo richbodo 644 Oct 12 16:42 sum.o
-rw-r--r-- 1 richbodo richbodo 67 Oct 12 15:57 testsum.rb
  • example_wrap.c - created by the swig command, this a c wrapper for your library with a bunch of additional wrapper functions that translate your functions and data into ruby callable functions and data. see chap 21 in the pickaxe.
  • Makefile - this is the makefile that was generated by extconf.rb. It will compile your library and that wrapper and make the mac bundle for you.
  • sum.o - the object file generated from sum.c
  • example_wrap.o - the compiled version of example_wrap.c
  • example.bundle - mac osx stores code in bundles, so make makes one and puts it in your site-ruby directory when you type "make install"
So that' it. You make that swig interface file, you call swig. Example_wrap.c gets created. You make the extconf.rb file and run it. A makefile gets created for you. You make your project by calling make and you are done.

References:
  • Chapter 21 in the pickaxe book.
  • The ruby swig page: http://www.swig.org/Doc1.3/Ruby.html#Ruby_nn5
  • The swig basics page: http://www.swig.org/Doc1.3/SWIG.html#SWIG_nn3
  • Apple Developer Docs: http://developer.apple.com/documentation/

Tuesday, October 09, 2007

Method for organizing volunteer efforts

The Cancer Beatdown

At it's simplest, it's just a task manager for researchers who would like to utilize volunteers in the search for a cure. For volunteers, it's a way to get their hands bloody at cancers expense. "No, you won't cure cancer, but maybe once in a while, you can come on over here and get a few good licks in". And, yes, I'm going to continue to refer to fighting cancer in more colorful ways than just "fighting cancer". It's quite satisfying.

No money, no donations of protons or neutrons. Just electrons, photons, and elbow grease. This isn't a site for beggin; it's just a site that invites others to get in on the beatdown while the beatin' is good.

The idea is based on the premise that the pool of smart, motivated volunteers is underused and underorganized by the cancer research community. In other words:

1) there are a lot of very smart people who would get a profound satisfaction from knowing that they contributed in some small way to developing a cure for cancer.

2) there are a lot of researchers who will buy that exposing some of their tasks in a sensical way to an army of smart people who want to work on them could speed their work.

One case in which this might work is:

A researcher needed to develop a significant new body of software, and was looking to task some portions of that development out to volunteers in a test-driven manner - "I write the tests, you guys make as many as you can pass before I do".

I recently participated in agile development of this sort, using pivotals excellent agile project manager, which is influencing my thinking on this type of application.

The basic site workflow for Cancer Beatdown is as expected:

1) Site manager invites a few researchers who are doing promising work.

2) Researchers post the task, whether it's stuffing envelopes on-site or coming up with a new way to search a massive database for correlations. The required form of the solution is also specified.

3) Volunteers propose solutions.

Some sensical reputation/core user policies would be adopted - i.e. Researchers or the site manager can invite more researchers, after vouching for them. Volunteers are invested in performance.

Personally, I would love to volunteer for some well-defined and promising projects in which I could invest one day per month. I know some people who would never consider such a thing, and some that would invest considerably more time. What do you think?