Before Bundler came to save us all from dependency hell, it was pretty common to check in specific versions of plugins and gems into a Rails app’s vendor directory. Unfortunately, due to our flawed nature as mortal developers, the temptation to make that one little change directly in the vendor’d code sometimes proved to be too great to master.

The result: forked and frozen code whose change history is deeply integrated into the entire project’s history, making upgrading nearly impossible.

But with Bundler and its integration with git there’s little reason for this. Your app can depend on your own private version of the gem. If you’re a good open source citizen, having a separate gem will make it easily to contribute your changes back.

Checkout this Gist for how to extract a plugin (in this case active_shipping) and preserve its changes. [One of these days I just need to move my blog over to GitHub and be done with it… but I digress…]

If your codebase is like mine was, the first commit included both the initial plugin import and a bunch of changes. If that’s the case, you’ll probably have some conflicts to resolve. But if you’re hear, hopefully you know how to do that.

Also, you might take the chance to rewrite your history to remove references to project specific things in comments so that when you push changes to the public gem the authors don’t have to pick through the accumulated history of years of your project’s secret-garden-development. Again, that’s something left to other tutorials.

Anyway, the most amazing part to me was git-filter-branch, which I learned about in this Stack Overflow question. A word of warning though: this tool can have some sharp edges. I think Patrick Thomson said it best: “filter-branch really should be renamed to nuclear-chainsaw considering how dangerous and powerful it is.

Ever had one of those problems that makes you feel like a complete idiot? You tinker and tinker and it’s still there. You recreate an entire configuration file from scratch trying to figure out the exact point where everything goes haywire.

Sorry, I should rephrase that first sentence: the solution makes you feel like an idiot.

Often the culprit is a misspelled variable or filename (which looked right every one of the last hundred times you looked at it!). The worst is when you realize you’ve been looking at cached content the whole time or the system has been running a stale version of your code for some reason.


We prideful IT-types usually move onto the next problem grumbling excuses and muttering curses, but not me! Not this time! I will help my fellow human beings who are desperately Googling for a solution!

So here’s what happened to me recently. I’ve been hacking on Github’s wonderful Gollum git-based wiki. As it is, Gollum is setup to run using a `bin/gollum` script. Nice for someone poking around with the project, but I wanted something that worked nicely with Capistrano and Passenger.

I extracted what I thought was a reasonable from the bin/gollum script:

require 'rubygems'
require 'bundler'
Bundler.require(:default, ENV['RACK_ENV'].to_sym)

options = {}
wiki_options = {}

require 'gollum/frontend/app'
Precious::App.set(:gollum_path, "wiki_data.git")
Precious::App.set(:wiki_options, wiki_options)! # <- SPOILER ALERT: this is what's wrong

What happened when I started this up? The damnedest thing: Passenger was trying to spin up WEBrick!

[Mon Mar 14 07:16:00 2011] [notice] Apache/2.2.16 (Ubuntu) Phusion_Passenger/3.0.5 configured -- resuming normal operations
[2011-03-14 07:16:07] INFO  WEBrick 1.3.1
[2011-03-14 07:16:07] INFO  ruby 1.9.2 (2011-02-18) [x86_64-linux]
[2011-03-14 07:16:07] INFO  WEBrick::HTTPServer#start: pid=30305 port=4567

What’s going on here? Is it a problem with Passenger? Apache (also tried Nginx)? Rack? Sinatra? Gollum? So many components!

Well, of course it’s my fault. You see, here’s the documentation for `!` (`Sinatra::Base` is, of course, the superclass of `Precious::App`, Gollum’s default frontend):

Run the Sinatra app as a self-hosted server using Thin, Mongrel or WEBrick (in that order)

So, it turns out `run!` is not just a convenience method for Rack::Builder#run… It actually tries to spin up a self-hosted server. Which is how I ended up in the bizarre situation of Passenger trying to start another server on port 4567.


For reference, here’s the working

require 'rubygems'
require 'bundler'
Bundler.require(:default, ENV['RACK_ENV'].to_sym)

require 'gollum/frontend/app'
Precious::App.set(:gollum_path, "wiki_data.git")
Precious::App.set(:wiki_options, {})

run Precious::App

UPDATE: (Jan. 13, 2011) As so often happens in the Ruby world, this blog entry is out of date. I’ve heard reports about various SOAP libraries no longer working with Ruby 1.9, but I haven’t had the need to test them out. If you’re working with Salesforce and Ruby, development dm-salesforce seems to have carried on, meaning that you won’t have to do most of these steps.

I’m going to leave this up for posterity and to help anyone who may need to dig around in the undocumented parts of any of these libraries. If you are one of those unlucky people, please understand that you have my condolences and best wishes for success.

Sooner or later it’ll probably happen to you: WSDL. SOAP. WS-Deathstar.

The horror.

You say, “I use Ruby! We’re all RESTful around here, man! My webapp consumes and produces YAML and JSON with no angle-bracket residue (it’s better for the environment). Our Kool-Aid’s delicious!”

I know — it really, really is.

A long time ago (yesterday), I thought I was safe. Then I had to integrate an internal webapp with Salesforce.

Me: “Do they have a way to do feeds? JSON? Anything?”

Internet: “SOAP.”

Me: “That’s for those Java and C# people! Not hip cats like me.”

Internet: “Try our SOAP. It’s delicious.”

It really, really wasn’t.

Someone asked me, “Why not use Activesalesforce?” Sure, some of these problems could have been avoided that way. However, my app doesn’t use Rails or ActiveRecord. I figured I was going enough against the grain that I should just strike out on my own. And WSDL makes it easy (or just possible), right?

So, as half cautionary tale / half survival guide, I give to you my tale of SOA-WOE. Presented below are the steps I went through to get Ruby and Salesforce’s WSDL to play along… well, not nicely, but at least… well, let’s just leave it at play.

…And before I get into it, let me preface by saying that a lot of my problems look silly now. The answer are, of course, in the documentation. But separating the wheat from the chaff requires knowledge that most people don’t have when they’re starting out tabula ras(o)a.

By the way, if you’re reading this for generic Ruby/WSDL and not Salesforce information, you can skim the first three steps.

1. Signup

As all hip web developers have, I’ve read Getting Real. So I learned that you’re supposed to lower the amount of effort it takes for a user to get on board. Having to sign up just to see the API docs and wiki is not in that vein. It’s sort of the first signpost on the road that we’re treading into unfamiliar territory. Well, I guess them’s the rules.

2. Get your Security Token

So yeah, here’s another thing that is not necessarily straightforward, but I guess there’s a reason in this case: you have to have a Security Token to append to your password when logging in through SOAP. Overall, it’s probably a good idea.

Didn’t notice that? Neither did I. But don’t feel so bad, neither did this guy.

To get your Security Token, go to the Setup link at the top of the page, find the “My Personal Information” section and click on the “Reset your security token” link. It’s around there somewhere.

3. Get your WSDL

Now look on the left column under “App Setup”. Click “Develop” and find the “Download your organization-specific WSDL” link.

Look at all the choices! Well, which one should you use?

You could do what I did:

  1. Stare at the options for a little while
  2. Say “Awh, fuck it.” at a volume that may or may not have been heard by my fellow cube-sters (it probably was).
  3. Download “Enterprise WSDL”. Because I’m feeling enterprisey.

4. Install the SOAP4R gem

From what I’ve gleaned, SOAP4R is the same as the SOAP standard library. By why not get up to date with the latest and greatest:

gem install soap4r

5. Run wsdl2ruby

The example linked above builds the SOAP driver at runtime, which can take awhile. His method is good for playing around, but you should probably just go ahead an build them.

wsdl2ruby.rb --wsdl ~/Desktop/enterprise.wsdl.xml --type client

Change your paths accordingly. This will produce 4 files in your current working directory: default.rb (which does something), defaultDriver.rb (which does something else), defaultMappingRegistry.rb (which also does something, but not the same thing as the other somethings), and SforceServiceClient.rb (which does nothing).

So you’ve got these files. Don’t edit them! Someday you will run that code generator again and you’ll come to rue it. Some even say don’t version control them. Instead they’d put the wsdl under version control and add a Rake task to run wsdl2ruby (this assumes that wsdl2ruby won’t break in the future and it installed on any future machines). Choose for yourself.

Are we there yet?

No, but it feels like we almost are, right? I was once like you.

Let me first give you some incomplete sample code:

require 'rubygems'
gem 'soap4r'                                     # require 'soap4r' doesn't work
require 'soap/soap'
require 'defaultDriver'                        # this is one of the generated files that does something.

# d = driver;  I used one letter names here because I'm playing around in IRB.
d =
d.wiredump_dev = STDOUT

# l = login result
l = d.login(:username => your_username, :password => your_password + security_token)


Should work, right?


6. Reset the Endpoint URL

The first error this code will produce is this:

SOAP::FaultError: UNKNOWN_EXCEPTION: Destination URL not reset. The URL returned from login must be set in the SforceService

This is simple enough to fix: after the login result (“l”) is returned, add d.endpoint_url = l.result.serverUrl

7. Maintain the Session ID

Now here comes the really fun part. The next error this code will produce is:

SOAP::FaultError: INVALID_SESSION_ID: Invalid Session ID found in SessionHeader: Illegal Session

Most of the search results in the old Google say this has something to do with the “Lock sessions to the IP address from which they originate” box being checked (which is located under “Administration Setup” -> “Session Settings”). This might be your problem, but in my case it was just that I didn’t know how to set headers in SOAP4R.

SOAP4R uses “header handlers” to modify outgoing and read incoming headers (and note that these are SOAP headers, not HTTP headers), so you’ll have to write one of your own (syntax highlighted version):

require 'soap/header/simplehandler'

class ClientAuthHeaderHandler < SOAP::Header::SimpleHandler 
  SessionHeader ="", "SessionHeader") 

  attr_accessor :sessionid 
  def initialize 
    @sessionid = nil 
  def on_simple_outbound 
    if @sessionid 
      {"sessionId" => @sessionid}

  def on_simple_inbound(my_header, mustunderstand) 
    @sessionid = my_header["sessionid"] 

I found this thread and and this presentation helpful in figuring this out.

Pulling it all together, here’s the finished product (syntax highlighted version):

require 'rubygems'
gem 'soap4r'
require 'soap/soap'
require 'defaultDriver'
require 'client_auth_header_handler'

d =
d.wiredump_dev = STDOUT

h =          # Create a new handler

l = d.login(:username => your_username, :password => your_password + security_token)

d.endpoint_url = l.result.serverUrl         # Change the endpoint to what login tells us it should be
h.sessionid = l.result.sessionId              # Tell the header handler what the session id is
d.headerhandler << h                          # Add the header handler to the Array of headerhandlers

d.getUserInfo("")                                 # Make an API call; the empty String appears to be necessary

defaultDriver and client_auth_header_handler (the code from above) are assumed to be in the current directory.

More API calls can be found in the API docs.

Lastly, I want to point out another resource that helped out with this process that I didn’t find a good place to link elsewhere in the article: Getting Started with SOAP4R, by Mark Thomas.


Ok, so a lot of these problems are because I was impatient to get started. There were API documents with Java examples I should have been reading! Sorry, I thought we were working on the Web here, where HTTP is king. I know HTTP. I know a couple of serialization formats. I should be good to go, right?

If you’re a seasoned SOA-type then a lot of what I’m complaining about probably seems like old hat to you. Sure, I’m whiny. Kids like me don’t know how good they have it!

Still, I can’t help thinking that there’s no reason for the extra overhead that comes with WSDL and SOAP. It might be that I work with dynamic languages most of the time. I miss my REST. I miss my YAML and JSON. I wanna go home.

Learning how to use FuseFS

January 29, 2008

Well, I started this blog about a year ago and haven’t written anything since. Ah, but now my writer’s block has been broken and here I am.

For a little while now, I’ve been interested in FUSE, but documentation has been scant and for the most part, inaccessible to me. There is even less documentation on Ruby’s bindings, FuseFS (a situation made worse by the existence of this song). The best example I’ve seen is Why the Lucky Stiff’s implementation of a filesystem for ActiveRecord objects.

Fortunately the package is reasonably well documented, and has some good sample code. Unfortunately, the business end of the library is in C, which complicates understanding what methods are doing what.

Here’s something that might help. We’re going to wrap all of methods of the supplied MetaDir (which mounts a Hash as a filesystem) in some logging calls. Now, I’m sure there’s a nicer way to do this, and you should blog about it.:P

   1  #!/usr/bin/env ruby
   2  require 'logger'
   3  require 'fusefs'
   5  METHODS_TO_IGNORE = [:__id__, :__send__, :send, :instance_variables, :puts]
   7  class TutorFS < FuseFS::MetaDir
   8    superclass.instance_methods.reject {|method_name|
   9      METHODS_TO_IGNORE.include? method_name.to_sym
  10    }.find_all {|method_name|
  11      method_name.to_s =~ /^[A-Za-z].*/
  12    }.each do |method_name|
  13      alias_method "old_#{method_name.to_s}".to_sym, method_name
  14      self.class_eval(%{
  15        def #{method_name}(*args, &block)
  16          @log.debug "Called #{method_name.to_s} with \#{args}"
  17          send(} + ":old_#{method_name}" +%{, *args, &block)
  18        end
  19      })
  21      puts "Wrapped :#{method_name}"
  22    end
  24    def initialize
  25      @log =
  26      @log.datetime_format = "%H:%M:%S"
  27      @log.level = Logger::DEBUG
  28      super
  29    end
  30  end
  32  if (File.basename($0) == File.basename(__FILE__))
  33    if (ARGV.size != 1)
  34      puts "Usage: #{$0} "
  35      exit
  36    end
  38    dirname = ARGV[0]
  40    unless
  41      puts "Usage: #{dirname} is not a directory."
  42      exit
  43    end
  45    root =
  47    # Set the root FuseFS
  48    FuseFS.set_root(root)
  50    FuseFS.mount_under(dirname)
  52 # This doesn't return until we're unmounted.
  53  end

When you mount this, it should give you access to the hash at the filesystem level. It should also start logging your messages and their arguments.