A few security releases ago, I converted my Rails 2.3 site to using bundler (and while I was at it, vendored all the gems I am using). This made subsequent security updates much easier. But now that Rails 4 is out, there will be no more security releases/backports for the Rails 2 series. And there have been some recent security releases for Ruby - which are only officially being added to Ruby 1.9 and above (though Heroku is continuing legacy support for 1.8.7 for a little while longer). Which makes it really, really past time to upgrade.

I intially tried installing ruby 1.9.3-p484 with my current rails 2.3.18 gems but I had trouble installing rmagick. RHEL 5 has an older version of ImageMagick so I need to use rmagick 1.15.17. To get rmagick to build on the new ruby I needed to move to RHEL6 so that I could get ImageMagick > 6.8.2 which I needed to move to rmagick 2.x. So I set up a quick Vagrant VM with CentOS 6.4 and got everything installed. However I had some trouble running my functional tests so I decided to fall back and try upgrading my rails version before upgrading my ruby. So as with all things Rails, the logical first step is Railscasts.

My development server has RVM on it. My default ruby is the ‘system’ ruby - REE-1.8.7-2009.10 with rails 2.3.18. Wth that combo, all my unit tests and cucumber features pass. So I installed rails-upgrade as a plugin, script/plugin install git://github.com/rails/rails_upgrade.git The plugin contains a task to backup some stock files that will be overwritten when I run rails new . in my CMS repository: rake rails:upgrade:backup

Then I need to set up Rails 3. I don’t want to deal with changing my REE in /opt so, I used the REE installed via RVM, ree-1.8.7-2011.03. I created a new gemset, rails-3.0 and installed bundler in it. Then changed the rails version in my Gemfile to 3.0.20 and ran bundle update. Then I upgraded rails using rails new ., accepting all of the requests to overwrite files. Then I started working my way through the changes I see with git status.

Ughhh horribly broken - can’t even run tests from this state.

Integration Testing AJAXy sites

A friend of mine was trying to write some integration tests against a site that makes a lot of use of AJAX requests for navigation. She started out using her standard tool, Watir, but was having trouble getting selectors th would work. Part of the issue was that she needed to explicitly wait for parts of the page to finish loading before looking for things in the DOM. She could fix that with some ‘wait until loaded’ stanzas. But the site she was testing didn’t have a lot of unique ids or classes. For one link she was trying to click, the only unique attribute on the link she wanted was in a data attribute - which watir wasn’t finding. I suspect there is a way to do it - but I can’t find where I can read about what watir uses to parse the DOM so I couldn’t help her figure out the correct syntax. So we decided to give capybara a try.

The capybara documentation says:

The capybara-webkit driver is for true headless testing. It uses QtWebKit to start a rendering engine process. It can execute JavaScript as well. It is significantly faster than drivers like Selenium since it does not load an entire browser.

It goes on to say we just ‘gem install capybara-webkit’. That gave me an error which the internet says can be fixed by installing Qt via ‘brew install qt’:

    Fetching: capybara-webkit-1.0.0.gem (100%)
    Building native extensions.  This could take a while...
    ERROR:  Error installing capybara-webkit:
            ERROR: Failed to build gem native extension.

        /Users/cnk/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
    Command 'qmake -spec macx-g++' not available

Worked fine.

Poking around Minitest and Capybara docs, I cobbled together the following:

    require 'minitest/autorun'

    require 'capybara'
    Capybara.default_driver = :selenium

    class RequestTest < MiniTest::Spec
      include Capybara::DSL

      def test_google
        visit('http://google.com/')
        page.has_selector?('div#searchform')
      end

    end

Which gives the followign error:

      1) Error:
    test_google(RequestTest):
    LoadError: Capybara's selenium driver is unable to load
    `selenium-webdriver`, please install the gem and add `gem
    'selenium-webdriver'` to your Gemfile if you are using bundler.

Installing that gem makes the tests run - but it launches a full on browser. Not sure that is what I wanted. I thought I could use capybara-webkit to do headless JS testing.

ALSO, did I want to be inheriting from MiniTest::Spec given that I am trying not to use ‘describe’ syntax but instead use the ‘define a method who’s name starts with test’ style.

Intro

Ruby and Rails are my web tools of choice. But I work in a software development group who largely work in Python with a collection of home-built libraries. So I really enjoyed Mike Leone’s “Python for Ruby Programmers” at last year’s LA Ruby Conference. His talk emphasized similaries between the two languages - which was helpful to me since I had gotten caught up in the small but annoying syntax differences - like semantic whitespace and the need for ‘self.’ and ‘()’ all over the place. With a more balanced perspective, I have been reexamining the pros and cons of the Python language.

Explicit: One of Python’s principles is: Explicit is better than implicit. One manifisation of that principle is having to type ‘self’ as the first argument when defining an instance methods (and ‘cls’ when defining class methods). Kind of annoying but whatever.

Scoping: In Ruby, the class keyword starts a new scope. So any variables created between ‘class’ and it’s matching ‘end’ are local to that scope. Most of the time this leads to the behavior one wants - but if you want something different, then you need to use some of Ruby’s metaprogramming tricks such as class_eval to get around that change of scope.

In Python, ‘class’ does not create a scope for the names used inside the bodies of methods. So when you want to refer to an instance’s attributes, you need to say ‘self.attribute_name’ rather than ‘attribute_name’ (or @attribute_name in Ruby). That’s annoying for the common taste of instance attributes. But the lack of new scope for classes makes it possible for instances to access class attributes using the same syntax you would use for an instance’s own properties. (The example below is adapted from “Python Essential Reference (4th edition)” by David M. Beazley)

    class Account(object):
        num_accounts = 0

        def __init__(self, name, balance):
            self.name = name
            self.balance = balance
            Account.num_accounts += 1

        def __del__(self):
            Account.num_accounts -= 1

        def inquiry(self):
            return self.balance


    $ python
    >>> from account import Account
    >>> Account.num_accounts
    0
    >>> a = Account('a', 100)
    >>> a.num_accounts
    1
    >>> Account.num_accounts
    1
    >>> c = Account('c', 300)
    >>> Account.num_accounts
    2
    >>> a.num_accounts
    2
    >>> #### but if I assign to the instances's attribute, then we get
        #### separate attributes - one for the class and one for the instance

    >>> a.num_accounts = 3
    >>> Account.num_accounts
    2
    >>> a.num_accounts
    3
    >>> a.__dict__
    {'balance': 100, 'name': 'a'}

Another interesting thing from this example is that in Python one can easily access methods that are called during the object’s life cycle. The double underscores in the method names tell you that you are getting kind of into the guts of an object, but you can get to them fairly directly. I tried rewriting the example above in Ruby. It’s quite easy to increment a class variable or class instance variable when you create an object. But where can you decrement it when an object is destroyed? Can one explicitly destroy an object in Ruby? Not directly. In fact, this StackOverflow thread suggests if you want to track all Account instances, you should use the WeakRef class from the Ruby standard library:

    def all_instances
       # this will vacuum out the dead references and return the remainder.
       @@weakrefs_to_vehicles = @@weakrefs_to_vehicles.select(&:weakref_alive?)
    end

    def total_vehicles
       all_instances.count
    end

Installing

I was trying to follow this DevOps Toolbox tutorial but I am running across some issues that seem like version conflicts. The instructions just say to create a Gemfile that specifies vagrant and veewee with no version numbers. This implies that the tutorial was written when vagrant was still primarily distributed as a gem. But, as detailed in my previous post, I have Vagrant 1.2.2 installed as a stand alone program. I tried just omitting vagrant from my gem file, but the gem version got installed anyway because veewee 0.3.7 lists “vagrant >= 0.9” as a runtime dependency.

I poked around trying to figure out if it was possible to use veewee with the newer version of vagrant. The Veewee Readme on Github indicates that I should probably clone from the repository rather than using the veewee gem. So I cloned veewee into the top level of my devops_toolbox directory (which gave me commit 4cf8acc7507d646f897dcd40bf06b2bcc961a6c0). Then I created a Gemfile that referenced this checkout and then ran ‘bundle install’ in my devops_toolbox gemset.

    # Gemfile
    source 'https://rubygems.org'
    gem "veewee", :path => "./veewee/"

The Veewee repository comes with a .rvmrc file which tries to help by setting ‘bundle exec’ aliases for veewee and irb. Not sure that helps

• and it leads to conflicts with the .rvmrc file in my devops_toolbox parent directory. I tried removing it but then when I ran ‘bundle install’ in devops_toolbox, I got the following error:

    Using veewee (0.3.7) from source at ./veewee/

    veewee at /Users/cnk/Code/devops_toolbox/veewee did not have a valid
    gemspec.  This prevents bundler from installing bins or native
    extensions, but that may not affect its functionality.

    The validation message from Rubygems was:
      [".rvmrc"] are not files

    Using bundler (1.3.0)
    Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.

Creating a basebox

The tutorial says to use ‘vagrant basebox templates’ but vagrant no longer supplies the templates. But I can now get the same list from veewee. Fedora 19 just came out at the beginning of the month and comes with Ruby 2.0.0. I am tempted but having already had version conflict problems getting this far, I think I am going to play it safe and build a Fedora 18 box with Ruby 1.9.3. I am not sure if I am going to end up with p327 (which is what originally shipped with Fedora 18) or if the install will automatically install p448 from the updates. We’ll see.

    $ veewee vbox define '32bit-fedora18' 'Fedora-18-i386' --workdir=/Users/cnk/Code/devops_toolbox
    The basebox '32bit-fedora18' has been successfully created from the template 'Fedora-18-i386'
    You can now edit the definition files stored in
    /Users/cnk/Code/devops_toolbox/definitions/32bit-fedora18 or build the
    box with:
    veewee vbox build '32bit-fedora18' --workdir=/Users/cnk/Code/devops_toolbox

Other useful resources

• http://cbednarski.com/articles/veewee/

Update VirtualBox

So I want to get started with Vagrant again. So before getting started, I updated my VirtualBox install to 4.2.12. My Ubuntu VM still works just fine. So now I can move on to Vagrant.

Installing Vagrant

The last time I played with Vagrant it was distributed as a Ruby gem but apparently that is now deprecated. Following the instructions on the Getting Started Guide I first found and removed old copies of vagrant that I had. All of them seemed to be installed in a ruby 1.9.3 patchset that I didn’t have any more, so I just deleted the gem directories for that ruby.

While I was looking at my rubies and gemsets, I went ahead and updated the gems in my default ruby’s global gemset. I was a little surprised that I have a minitest gem installed (Doesn’t minitest ship with ruby now?) and that I appear to have quite an old version (Aug 27, 2011 according to the version list on rubygems.org). Curiouser and curiouser.

    bundler (1.3.0 < 1.3.5)
    io-console (0.3 < 0.4.2)
    json (1.5.5 < 1.7.7)
    minitest (2.5.1 < 4.7.1)
    rake (10.0.3 < 10.0.4)
    rdoc (3.9.5 < 4.0.1)
    rvm (1.11.3.6 < 1.11.3.7)
    slop (3.4.3 < 3.4.4)

Anyway, back to Vagrant. I downloaded the dmg for v1.2.1 and ran the installer. I see a Vagrant folder in my Applications folder - and /usr/bin/vagrant is available and shows the expected version number.

Making a VM

I have some other VirtualBox VMs installed in ‘/Users/cnk/VirtualBox VMs’ So it seems logical to add more there. I have a kind of mix of stuff in that directory so at some point I need to go back and figure out what pieces go with what other pieces. For now, let’s make a new directory for my new box, and initialize vagrant:

    $ cd /Users/cnk/VirtualBox\ VMs
    $ mkdir edx
    $ cd edx
    $ vagrant init

To create a VM I need a ‘box’ to use as a base image. The default in the Vagrant docs is precise32 which is an Ubuntu image. I may have to learn to like Ubuntu since last time I looked, the base VirtualBox images available for Ubuntu were a LOT smaller than the RHEL/CentOS ones I am more familiar with. ‘vagrant box list’ actually turns up a lucid32 image from ???? Apparently vagrant stores the boxes in ~/.vagrant.d/ by default. I see that directory - with a boxes/lucid32 subdirectory. However, I also have a directory, ~/.vagrant. From the moddates and boxes listed, this is from the first time I played around with Vagrant (for setting up sandboxes to run Brakeman in). If I ever need those images again, I may need to figure out if I can just move them. But for now, I am just leaving them alone.

I am a little disappointed that vagrantup.com doesn’t have a semi-authoritative list of available boxes. The list at http://www.vagrantbox.es/ seems to still be being maintained. But I think I’ll start off by just downloading the 32 bit Ubuntu Precise Pangolin image from VagrantUp:

    $ vagrant box add precise32 http://files.vagrantup.com/precise32.box

I edited my edx/Vagrantfile to reference the precise32 image and then did ‘vagrant up’. It booted - pretty quickly. I can ssh in with ‘vagrant ssh’ and then turn the box back off with ‘vagrant halt’. That’s enough for now. Tomorrow, pick puppet or chef and create a working python environment.