One of the big changes in Rails 3 is the move to “everything is an engine”.  You end up reading a lot about this in relation to plugins, mentioning how this architecture makes it so you can easily embed entire rails applications into others since they’re all just derivatives of Railties.  While this holds true for very simple cases, you’ll quickly find simply creating a rails app, and sticking it in vendor/plugins doesn’t work the way you’d expect from reading about it.  You really do need to create your plugin in a certain way to actually make it work.  Over here is a good writeup on creating a Rails Engine based plugin installed as a gem.

Serving Static Assets

Now, serving static assets from your plugin is a bit on the unintuitive side, and likely for a good reason.  You generally do not want to do this for any kind of production system. However, like most rules, there are times when breaking them makes pragmatic sense.  In this use case we have an internal tool that we want to be able to easily extend.  The maximum number of concurrent users will likely be in low teens, and serving static assets through the rails app is a non-issue.  Extending the app includes providing views, controllers, models, routes, and images.  The first three are extremely straightforward and intuitive, routes use a slightly different enclosing syntax in an Engine, but are otherwise identical.  Images, however are a bit tricky.

There are two ways to create an Engine based plugin, and they are a bit incompatible with each other.  You can either install your plugin as a gem, or as a plugin in the app.  In both cases, you are going to use the ActionDispatch::Static middleware to serve your content. I am also assuming that you are placing your content in the “public” directory in the root of your plugin’s file structure.

Gem Based Plugin

A gem based plugin is initialized via the definition of itself in your plugin’s lib directory.  You can see an example of this below.  In order to set it up correctly you should put your middleware line in an initializer in your declaration:

module MyEngine
  class Engine < Rails::Engine
    initializer "static assets" do |app|
      app.middleware.use ::ActionDispatch::Static, "#{root}/public"
    end
  end
end

In App Plugin

Unfortunately, if the above file exists in a plugin installed to vendor/plugins your app will fail to load.  This is because rails automatically assumes all plugins are Engines, and initializes them as instances of the Plugin class which inherits from Engine.  It will autoload everything in lib, see the declaration, and then fail out with the incredible error message “[Your Plugin] is a Railtie/Engine and cannot be installed as plugin”. Which is somewhat misleading since a plugin is an Engine, just not explicitly.

To get around this issue, you just have to move where you do your initialization and setup. Inside your plugin’s init.rb (which should be in the root of your plugin’s file structure) is where you can do this kind of stuff.  Rails boots your Plugin, and then loads this file with the variable “config” set for you to do your work.  So it’s simply:

config.middleware.use ::ActionDispatch::Static, "#{root}/public"

Load Order

I strongly recommend using middleware.use as it will place your middleware after the parent app, and therefore will cause your plugins assets to be of lower priority than everything else in the stack. This means that files in the parent app’s public directory will load instead of the plugin’s in the case of a conflicting name.  You still have options though.  To give your plugin’s static files priority over everything except the parent app’s static files, use:

middleware.insert_after ::ActionDispatch::Static, ::ActionDispatch::Static, "#{root}/public"

To give your plugin top priority in static asset serving use:

middleware.insert_before ::ActionDispatch::Static, ::ActionDispatch::Static, "#{root}/public"

The first was how to test the service based apps outside of any consuming application. The specific one in question did not present any HTML pages at all, and only returned Javascript that was executed. A few ideas were floated around like dev/test only controllers or basic one-off apps that were specifically for testing the service. Eventually we cam across a solution that worked and seemed much cleaner. We combined the one-off app idea with rack middleware to create an application within our app that was only loaded in certain environments. We used Sinatra to make a simple application that would respond with test pages on routes prefixed with ‘/test’, and then added it with config.middleware.use. Combining this with Celerity and Capybara gave us a very clean testing solution for an application that only served Javascript.

There is one gotcha to this, and it only involves using Sinatra as middleware in Rails. Rails attempts to determine if the middleware you gave it is a proc, and in doing so prevents you from being able to use any middleware that can respond to Rack requests as a singleton (like Sinatra). Our quick fix was to monkey patch Sinatra to remove self.call, but a more permanent solution is fixing the way Rails 2.3 does its middleware. I hope to submit some patches for that soon.

1
2
3
4
5
6
7

module Sinatra
  class Base
    class << self
      remove_method :call
    end
  end
end

Our second hurdle was determining how to test our consuming apps using Cucumber. The main problem we had is that we use reverse proxying to make the services appear to be coming from the same domains as the rest of the apps. This was easily taken care of in manual tests, where we could simulate our load balancers using Apache or nginx, however with automated tests, this becomes a problem. The automated test suite doesn’t run behind a proxying layer or a web server, it just hits the app directly. So, tackling that problem, I am releasing a gem I developed called rack-reverse-proxy. What this gem does is allow you to put reverse proxy rules directly into your middleware stack so they are independent of your web server. This is probably not a good idea for production systems, load balancers and web servers are much better at this sort of thing, and due to the way rack works I have to wait until the entire request is received before forwarding it. However it’s great for development and testing, allowing you to test your app directly without the need of an intermediate web server layer.

1
2
3
4
5
6
7
8
9

  require 'rack/reverse_proxy'
 
  use Rack::ReverseProxy do
    # Forward the path /test* to http://example.com/test*
    reverse_proxy '/test', 'http://example.com/'
 
    # Forward the path /foo/* to http://example.com/bar/*
    reverse_proxy /^\/foo(\/.*)$/, 'http://example.com/bar$1'
  end

$ find . -name ".git"  -type d -prune -execdir git pull \;

This is a little command line snippet that will search for git repositories in all of the subdirectories of wherever you run it and update them.  Really handy in this case, because you can update all of your bundles at once, but it will work in any case where you have cloned a bunch of repos and want them to all stay up to date.

Rails catches *any* problem loading the Rails gem, and then responds with that error.

 rescue Gem::LoadError => load_error
      $stderr.puts %(Missing the Rails #{version} gem. Please `gem install -v=#{version} rails`, update your RAILS_GEM_VERSION setting in config/environment.rb for the Rails version you do have installed, or comment out RAILS_GEM_VERSION to use the latest version installed.)
      exit 1

If you inspect load_error from the line above, you should see the actual error message. In my case it was that ActiveSupport required Rack v1.0.0, but Passenger had already instantiated Rack v1.1.0. This led to a failure when loading the Rails gem, which was of course incorrectly reported. My quick solution was to uninstall the new version of rack, and force the install of the older version.

As a side note: I have the world’s worst code highlighter.

The application demonstrates a few app engine concepts.  In models.rb you’ll see a sharded counter implementation using the direct Datastore API, and a traditional model using Datamapper as your ORM (ODM?) to Datastore.  The counter also demonstrates usage of the Memcache API.  Inside app.rb you can see some simple usage of the Users API, including how to check to see if a user is logged in, present a user with a login screen, and interacting with the object.  Lastly, you will find an example of how to determine your app’s name and version in config.ru.

Following the presentation I received a number of requests for my slides, so they are included below.  Slide 13 illustrates pushing your own app to App Engine.  As I mentioned in the talk, I strongly suggest setting aside one of your app ids for testing new ideas, since you only get 10 ids total.