So I have been fooling with chef for nearly a year now - including going to an Opscode workshop before Rocky Mountain Ruby - but I haven’t really gotten over the hump into using it for real. But I think it’s time.

So starting with the Quick Start guide on LearnChef, I set up my laptop as the workstation. My default ruby under RVM is currently 2.0.0-p353. I created a gemst called chef and installed the chef gem (current version is 11.8.2). During the Chef workshop we all signed up for the free hosted chef server, so I have a login (ckiser) and organization (ckiser) already. I downloaded the starter kit. (It warned me that it would reset my user and organization keys but that’s probably fine.) I moved the chef-repo directory into my home directory. The README said that stuff there should be under version control, so I did git init and committed all the files before starting.

Chef Quick Start - with Vagrant and Chef Server

First off, we installed 2 cookbooks from the Opscode community site, apt and apache2, with knife cookbook site install [repo]. I am a little unclear on what ‘site’ means in that command but anyway, that command created 2 new directories in my cookbooks directory. Then we created our own cookbook. First we installed a dependency, magic_shell. Then we generated the cookbook: knife cookbook create aliases, added the dependency line in the metadata.rb file, and then edited the default recipe. The tutorial then told us to upload the cookbooks to our hosted chef using knife cookbook upload --all. It uploaded all the cookbooks - the ones we got from the community site and the aliases cookbook we just made - and all without our having to commit them to git.

But I do have one mystery. The cookbooks we installed using knife cookbook site install [some repo] do not show up as untracked files for git - but the aliases cookbook we just created ourselves does. I don’t see anything obvious like mentions of the community cookbooks in a .gitignore or .chefignore file. So how does git know the difference?

    # Untracked files:
    #       cookbooks/aliases/

Then the guide had us boot a VM from the Vagrant file that came with the starter kit - using an Ubuntu image we downloaded from OpsCode. The file name, opscode_ubuntu-12.04-i386_provisionerless.box, makes me think that it doesn’t have chef installed, nor any other ruby. When I did the initialization that the guide suggested, I see output that appears to first install chef from OpsCode’s s3 stash:

    ~/chef-repo) $ knife bootstrap localhost --ssh-user vagrant \
                                             --ssh-password vagrant \
                                             --ssh-port--2222 \
                                             --run-list "recipe[apt],recipe[aliases],recipe[apache2]"
                                             --sudo

    localhost Downloading Chef 11.8.2 for ubuntu...
    localhost downloading https://www.opscode.com/chef/metadata?v=11.8.2&prerelease=false&p=ubuntu&pv=12.04&m=i686
    localhost   to file /tmp/install.sh.1049/metadata.txt
    localhost trying wget...
    localhost url https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/i686/chef_11.8.2-1.ubuntu.12.04_i386.deb
    ...
    localhost Installing Chef 11.8.2
    localhost installing with dpkg...
    localhost Selecting previously unselected package chef.
    localhost Unpacking chef (from .../chef_11.8.2_i386.deb) ...
    localhost Setting up chef (11.8.2-1.ubuntu.12.04) ...
    localhost Thank you for installing Chef!
    localhost Starting Chef Client, version 11.8.2
    localhost Creating a new client identity for ckiser-starter using the validator key.
    localhost resolving cookbooks for run list: ["apt", "aliases", "apache2"]
    localhost Synchronizing Cookbooks:..

Hmmm so perhaps I don’t have to preinstall ruby and chef. The bootstrap process can install the latest version of chef (11.8.2) packaged with an up to date ruby, ruby 1.9.3p484 (2013-11-22 revision 43786) in /opt/chef/embedded/bin/ruby.

Also note that the bootstrap command, after installing chef, registered the node we are operating on with my hosted chef server. The name, ckiser-starter, appears to come from the Vagrant file that I downloaded at the start of the tutorial, config.vm.hostname = "ckiser-starter". Are nodes machines? or types of machines? I think the later, but not sure. I would like to be able to test out my configuration changes on a local VM before pushing them to my Linode. So is that 1 node and 2 environments? or 2 nodes? I think the former but am not 100% sure.

I have been sharing hosting with some old work collegues for a long while but I am getting the urge to play with some Devops tools. I don’t want to disrupt them so I need another sandbox. I did most of the system admininstration for our Rails Rumble team the last couple of years and thought the Linode VPSs were reasonably speedy and their configuration web site not too crazy making. And I suspect for the things I have been doing, their lowest end package will be just fine.

First decision: what Linux distribution do I want to run? I am more familiar with RPM-based boxes so am tempted to go with either their Fedora 19 or CentOS 6.4 options. However, the docs for both of those consider 1G RAM to be a bare minimum for any install. While Ubuntu quotes 0.5G as their minimum. I don’t know if the difference is a difference in expectations (is slow acceptable?) but I rather suspect that Ubuntu is a bit trimmer. So time to suck it up and learn to like the Debian package management stuff.

This is a server that I want to be pretty stable, so I think the Precise Penguin 12.04 LTS distribution is what I want. I tried doing a couple of trial installs from StackScripts and one just from the configuration manager and all of them installed and then refused to boot for “unknown reasons”. Not good. I tried booting one in rescue mode and logging in via the lish console. But I am not really sure if there is any log file that would show me any information. I don’t recall exactly where to look but I didn’t see anything while poking around in things in /var/log/*. One theory is that my problem was choosing the 32bit distro. Once I capitulated and asked for a 64bit system, I could build and boot in a variety of ways, just from the distro, or from either of the following StackScripts:

1. Minimal Chef Solo which upgrades all the currently installed packages, then adds the Opscode apt repository and installs chef from there.

2. Ubuntu 12.04 - Ruby 1.9.3 which installs the packages needed to compile ruby from source, and then does just that (Ruby 1.9.3-p194). Or at least it says it does that. I saw evidence that it had done the downloading and untarring but ‘which ruby’ gave nothing. Following the steps listed by hand worked just fine so I am not sure if there was a timing issue or perhaps something different about running the commands interactively or what. Anyway mostly works.

OK so what strategy do I want to use? The first one seems generally easier but installs an older ruby (1.8.7-p352) and older chef (10.18.2) into /usr/bin. The second one gives me a recent ruby (and I can make my own script just like it for the latest ruby 1.9.3 version). It also installs into /usr/bin so it will be the global system ruby. But it is probably good enough to be the ruby I use for my Rails hosting - without bothering with RVM. Hmmmm let’s look at setting up Chef before making my final decision.

A few security releases ago, I converted my Rails 2.3 site to using bundler (and while I was at it, vendored all the gems I am using). This made subsequent security updates much easier. But now that Rails 4 is out, there will be no more security releases/backports for the Rails 2 series. And there have been some recent security releases for Ruby - which are only officially being added to Ruby 1.9 and above (though Heroku is continuing legacy support for 1.8.7 for a little while longer). Which makes it really, really past time to upgrade.

I intially tried installing ruby 1.9.3-p484 with my current rails 2.3.18 gems but I had trouble installing rmagick. RHEL 5 has an older version of ImageMagick so I need to use rmagick 1.15.17. To get rmagick to build on the new ruby I needed to move to RHEL6 so that I could get ImageMagick > 6.8.2 which I needed to move to rmagick 2.x. So I set up a quick Vagrant VM with CentOS 6.4 and got everything installed. However I had some trouble running my functional tests so I decided to fall back and try upgrading my rails version before upgrading my ruby. So as with all things Rails, the logical first step is Railscasts.

My development server has RVM on it. My default ruby is the ‘system’ ruby - REE-1.8.7-2009.10 with rails 2.3.18. Wth that combo, all my unit tests and cucumber features pass. So I installed rails-upgrade as a plugin, script/plugin install git://github.com/rails/rails_upgrade.git The plugin contains a task to backup some stock files that will be overwritten when I run rails new . in my CMS repository: rake rails:upgrade:backup

Then I need to set up Rails 3. I don’t want to deal with changing my REE in /opt so, I used the REE installed via RVM, ree-1.8.7-2011.03. I created a new gemset, rails-3.0 and installed bundler in it. Then changed the rails version in my Gemfile to 3.0.20 and ran bundle update. Then I upgraded rails using rails new ., accepting all of the requests to overwrite files. Then I started working my way through the changes I see with git status.

Ughhh horribly broken - can’t even run tests from this state.

Integration Testing AJAXy sites

A friend of mine was trying to write some integration tests against a site that makes a lot of use of AJAX requests for navigation. She started out using her standard tool, Watir, but was having trouble getting selectors th would work. Part of the issue was that she needed to explicitly wait for parts of the page to finish loading before looking for things in the DOM. She could fix that with some ‘wait until loaded’ stanzas. But the site she was testing didn’t have a lot of unique ids or classes. For one link she was trying to click, the only unique attribute on the link she wanted was in a data attribute - which watir wasn’t finding. I suspect there is a way to do it - but I can’t find where I can read about what watir uses to parse the DOM so I couldn’t help her figure out the correct syntax. So we decided to give capybara a try.

The capybara documentation says:

The capybara-webkit driver is for true headless testing. It uses QtWebKit to start a rendering engine process. It can execute JavaScript as well. It is significantly faster than drivers like Selenium since it does not load an entire browser.

It goes on to say we just ‘gem install capybara-webkit’. That gave me an error which the internet says can be fixed by installing Qt via ‘brew install qt’:

    Fetching: capybara-webkit-1.0.0.gem (100%)
    Building native extensions.  This could take a while...
    ERROR:  Error installing capybara-webkit:
            ERROR: Failed to build gem native extension.

        /Users/cnk/.rvm/rubies/ruby-2.0.0-p247/bin/ruby extconf.rb
    Command 'qmake -spec macx-g++' not available

Worked fine.

Poking around Minitest and Capybara docs, I cobbled together the following:

    require 'minitest/autorun'

    require 'capybara'
    Capybara.default_driver = :selenium

    class RequestTest < MiniTest::Spec
      include Capybara::DSL

      def test_google
        visit('http://google.com/')
        page.has_selector?('div#searchform')
      end

    end

Which gives the followign error:

      1) Error:
    test_google(RequestTest):
    LoadError: Capybara's selenium driver is unable to load
    `selenium-webdriver`, please install the gem and add `gem
    'selenium-webdriver'` to your Gemfile if you are using bundler.

Installing that gem makes the tests run - but it launches a full on browser. Not sure that is what I wanted. I thought I could use capybara-webkit to do headless JS testing.

ALSO, did I want to be inheriting from MiniTest::Spec given that I am trying not to use ‘describe’ syntax but instead use the ‘define a method who’s name starts with test’ style.

Intro

Ruby and Rails are my web tools of choice. But I work in a software development group who largely work in Python with a collection of home-built libraries. So I really enjoyed Mike Leone’s “Python for Ruby Programmers” at last year’s LA Ruby Conference. His talk emphasized similaries between the two languages - which was helpful to me since I had gotten caught up in the small but annoying syntax differences - like semantic whitespace and the need for ‘self.’ and ‘()’ all over the place. With a more balanced perspective, I have been reexamining the pros and cons of the Python language.

Explicit: One of Python’s principles is: Explicit is better than implicit. One manifisation of that principle is having to type ‘self’ as the first argument when defining an instance methods (and ‘cls’ when defining class methods). Kind of annoying but whatever.

Scoping: In Ruby, the class keyword starts a new scope. So any variables created between ‘class’ and it’s matching ‘end’ are local to that scope. Most of the time this leads to the behavior one wants - but if you want something different, then you need to use some of Ruby’s metaprogramming tricks such as class_eval to get around that change of scope.

In Python, ‘class’ does not create a scope for the names used inside the bodies of methods. So when you want to refer to an instance’s attributes, you need to say ‘self.attribute_name’ rather than ‘attribute_name’ (or @attribute_name in Ruby). That’s annoying for the common taste of instance attributes. But the lack of new scope for classes makes it possible for instances to access class attributes using the same syntax you would use for an instance’s own properties. (The example below is adapted from “Python Essential Reference (4th edition)” by David M. Beazley)

    class Account(object):
        num_accounts = 0

        def __init__(self, name, balance):
            self.name = name
            self.balance = balance
            Account.num_accounts += 1

        def __del__(self):
            Account.num_accounts -= 1

        def inquiry(self):
            return self.balance


    $ python
    >>> from account import Account
    >>> Account.num_accounts
    0
    >>> a = Account('a', 100)
    >>> a.num_accounts
    1
    >>> Account.num_accounts
    1
    >>> c = Account('c', 300)
    >>> Account.num_accounts
    2
    >>> a.num_accounts
    2
    >>> #### but if I assign to the instances's attribute, then we get
        #### separate attributes - one for the class and one for the instance

    >>> a.num_accounts = 3
    >>> Account.num_accounts
    2
    >>> a.num_accounts
    3
    >>> a.__dict__
    {'balance': 100, 'name': 'a'}

Another interesting thing from this example is that in Python one can easily access methods that are called during the object’s life cycle. The double underscores in the method names tell you that you are getting kind of into the guts of an object, but you can get to them fairly directly. I tried rewriting the example above in Ruby. It’s quite easy to increment a class variable or class instance variable when you create an object. But where can you decrement it when an object is destroyed? Can one explicitly destroy an object in Ruby? Not directly. In fact, this StackOverflow thread suggests if you want to track all Account instances, you should use the WeakRef class from the Ruby standard library:

    def all_instances
       # this will vacuum out the dead references and return the remainder.
       @@weakrefs_to_vehicles = @@weakrefs_to_vehicles.select(&:weakref_alive?)
    end

    def total_vehicles
       all_instances.count
    end