I got a call today from someone who had upgraded their desktop from Ubuntu 8 to Ubuntu 12 and was having trouble getting Ruby reinstalled. She had previously installed Ruby from source (as apparently everyone needs to do in the Debian/Ubuntu world) without any problem - but not this time. I do a reasonable amount of Ruby and system administration but I work in a RedHat shop so don’t know a ton about Debian-derived distros. But I thought I would try to see what I could do to help.

Installing Ubuntu 12.04

First thing I need is an Ubuntu box to practice on. I had VirtualBox installed and have used Vagrant to manage some boxes. However, since Vagrant is written in Ruby and usually does its configuration of the box with Chef or Puppet, I think the premade vagrant images will already have Ruby installed. - making it difficult to figure out the correct steps for installing Ruby from source. So I think I should install Ubuntu without my favorite virtualization tool.

First, I upgraded my VirtualBox software to 4.2.4. (My older boxes appear to still run but they appear to have lost their Guest Additions; I’ll have to go back and sort that out).

Ubuntu is pretty common so I searched for a prebuilt (hopefully minimal) VirtualBox image and found a promising looking one at http://virtualboxes.org/images/ubuntu/. I downloaded this one:

    Ubuntu Linux 12.04 x86
    Size (compressed/uncompressed): 769 MB/3.2 GB
    Link: http://sourceforge.net/projects/virtualboximage/files/Ubuntu%20Linux/12.04/ubuntu_12.04-x86.7z
    Active user account(s) (username/password): ubuntu/reverse
    Notes: Guest Additions NOTinstalled; tip: set Video RAM 64MB minimum

First issue: that is compressed with 7zip so I’ll need p7zip to uncompress it. Fortunately I have Homebrew installed so I just did:

    $ brew install 7zip

This installed a formula it said was p7zip from https://github.com/mxcl/homebrew/commits/master/Library/Formula/p7zip.rb and brew list p7zip shows:

    $ brew list p7zip
    /usr/local/Cellar/p7zip/9.20.1/bin/7zr
    /usr/local/Cellar/p7zip/9.20.1/bin/7za
    /usr/local/Cellar/p7zip/9.20.1/bin/7z
    /usr/local/Cellar/p7zip/9.20.1/lib/p7zip/ (6 files)
    /usr/local/Cellar/p7zip/9.20.1/share/doc/ (50 files)
    /usr/local/Cellar/p7zip/9.20.1/share/man/ (3 files)

After a little fishing around I think the uncompress command is ‘7z x '

    $ man 7z
    $ 7z x Downloads/ubuntu_12.04-x86.7z
    Extracting  ubuntu_12.04/ubuntu_12.04.vbox

This produces a folder named ubuntu_12.04 containing a small .vbox file and 3.5 GB .vdi file. The later is the ‘virtual disk image’ file that we will want to use to create our new VM. I followed [these instructions] (http://www.thelinuxdaily.com/2010/02/how-to-setup-a-pre-built-virtualbox-guest-image-tutorialguide/) and created a new VM with 512 MB RAM and using the pre-existing .vdi file. Then I just start the newly created VM from the menu.

There are a couple of peculiarities - the main one is that the requested keyboard is Italian. I found the keyboard settings and chose an English keyboard layout instead. Then I changed the update server to the server for the US and upgraded packages. The GUI says it wants to do 489 updates!

VirtualBox Guest Additions

Oh, and I don’t need the Guest Additions for preventing the VirtualBox from capturing my mouse but I do need them so I can cut and paste between the host and guest OSs. Under the VirtualBox Devices menu there is an item for Install Guest Additions. Clicked that to get them installed. However even though I rebooted the virtual machine, I still can’t copy and paste between the VM to my Mac.

Installing Ruby

OK now for installing Ruby. The person I am helping was installing Ruby globally from source but I wanted to see if RVM could install Ruby 1.8.7 for me. First I installed rvm:

    $ curl -L https://get.rvm.io | bash

That added a .bash_profile to my home directory. That didn’t play well with my shell in emacs so I moved that line to my .bashrc file. Then I installed ruby and made it my default ruby:

    $ rvm install 1.8.7
    $ rvm use --default 1.8.7

That worked just fine - but I can’t install any gems.

    $ gem install bundler
    ERROR: Loading command: install (LoadError)
           no such file to load -- zlib
    ERROR: While executing gem ... (NameError)
           uninitialized constant Gem::Commands::InstallCommand

Sounds like I am missing some dependencies. Fortunately RVM will tell me what it needs - and in Ubuntu syntax:

    $ rvm requirements
    $ sudo apt-get install build-essential openssl libreadline6
         libreadline6-dev curl git-core zlib1g zlib1g-dev libssl-dev
         libyaml-dev libsql3-dev sqlite3 libxml2-dev libxslt-dev
         autoconf libc6-dev ncurses-dev automake libtool bison
         subversion pkg-config

However, once I installed those, I still wasn’t able to install any gems. Apparently I should have installed the prerequisites BEFORE I installed the ruby. Once I got the order correct, installing ruby 1.8.7 via RVM also installed 4 gems:

    bundler
    rake
    rubygems-bundler
    rvm

For the last month or so the LA Ruby Study group has been reading “Building Awesome Command-Line Applications in Ruby”. For the most part it is a pretty good book and I think it makes a good intro book for folks who are new to Ruby, new to programming, and new to Unix. I get the sense that not everyone in the group is enjoying the tour of ‘the Unix way’ as much as I am but perhaps someday they will have reason to need ‘man’ or pipe (‘|’) and will remember that they heard about that in study group.

The examples the author chose - a script to backup MySQL databases and a to do application - are pretty good: small enough to be understood in 5 minutes but with plenty of room to add features, make tweaks, improve things in some small way. I am particularly enjoying the to do app because I haven’t written a command suite before. Judging from dependencies I have see while installing other gems, the thor gem seems to be a more popular then the GLI gem the author chose but I am pretty impressed with the breadth of support GLI’s scaffold created for us. For example the scaffold generates a Rakefile with a nicely tailored task for building Rdoc documentation. Before I found that, I had tried just ‘rdoc todo’ but that tried process every file in the directory - including several that had no reason to be in my Rdocs. I could have read the rdoc docs and figured out how to document only the project fieles I need to, but the generated rake task is a LOT easier - and gave me some examples of parameters I might want to set to customize further.

This week we are discussing how to distribute your code - as a Gem, an RPM, and as source on GitHub. The generated gemspec looks pretty straightforward and the gem builds and installs. And there are instructions on how to distribute opensource gems on rubygems.org and how to run a private gem server if you need to do that.

The author then goes on to repackaging your gem as an RPM using gem2rpm. His instructions seem to work - though these are a bit more concise. The generated RPM spec file looks reasonable - except for the fact that the development/testing dependencies are listed as full on ‘Requires’ when they are not actually needed to run the program. This Stack Overflow post notes the same issue and suggests a patch to gem2rpm but for just a handful of things to make into RPMs, it may be just as satisfactory to remove the development dependencies by hand while you are inspecting the RPM spec file.

A bigger issue is the value it automatically picked up for ‘gemdir’; it is my personal .rvm gemset directory. I wonder what the correct answer should be. If I just need to match where the rubygems installed via RPM expects gems, then I can probably hardcode a guess. But that seems less than ideal. I see that gem2rpm tries to calculate the correct value using:

    %global gemdir %(ruby -rubygems -e 'puts Gem::dir' 2>/dev/null)
    %global geminstdir %{gemdir}/gems/%{gemname}-%{version}

Poking around in the gem2rpm gem README, I found a link to the official packaging guidelines. I’ll have to read that and see if there is an official resolution for this.

This week’s Ruby Weekly was sponsored by AppFog. It sounded interesting so I thought I would give it a whirl. Their pricing model is interesting - it is based on how much RAM you use (free if you use less than 2 GB!). Seems sensible to me since that is what I have usually found to be the limiting factor for the kinds of sites I run. And I’ll have to say that their “Jumpstart” setup couldn’t have been easier - two clicks and a name for the app and I had a Rails app running in the cloud in well under 2 minutes. Of course all it does is show the “Welcome to Rails” page we all know and love. So now how do I make this useful?

If I were not a document-reading sort of person, it would have taken me less time to figure things out. When you are logged in, your AppFog console page is a list of your apps. Clicking on the app name gets you to “Mission Control” for the app and one of the tabs on that interface is “Update Source Code”. That page provides the details about how to install the ‘af’ command-line tool that were missing from the AF CLI doc page I had looked at first. It confirmed my suspicions that ‘af’ was distributed as a gem (even though all the docs did was link to the GitHub repository). I will probably want to use this for more than one project, so I’ll want ‘af’ available in all my gemsets. So I did:

    $ rvm gemset use global
    $ gem install af
    Successfully installed json_pure-1.6.7
    Successfully installed rubyzip-0.9.9
    Successfully installed mime-types-1.19
    Successfully installed rest-client-1.6.7
    Successfully installed terminal-table-1.4.5
    Successfully installed interact-0.4.6
    Successfully installed addressable-2.2.8
    Successfully installed uuidtools-2.1.3
    Successfully installed rb-readline-0.4.2
    Successfully installed af-0.3.18.2

Once past that hurdle, things look better:

    $ af login
    $ af apps

    +-------------+----+---------+--------------+-----------------+----+
    | Application | #  | Health  | URLS         | Services        | In |
    +-------------+----+---------+--------------+-----------------+----+
    | cnk         | 1  | RUNNING | cnk.rs.af.cm | cnk-mysql-29232 | rs |
    +-------------+----+---------+--------------+-----------------+----+

OK now for that documentation on actually using the tool.

I actually ended up using a combination of methods. I used the “Download Source Code” button on the web interface to get a copy of the generated code that is on my running site. I swapped out the “Welcome to Rails” page for my boring test message and then used ‘af’ to redeploy the new code:

    $ af update cnk --path appfog-test
    af update cnk --path appfog-test
    Uploading Application:
      Checking for available resources: OK
      Packing application: OK
      Uploading (38K): OK
    Push Status: OK
    Stopping Application 'cnk': OK
    Staging Application 'cnk': OK
    Starting Application 'cnk': OK

One odd thing, I couldn’t run any ‘af’ commands if I was in the directory of Rails code I downloaded from AppFog. Any ‘af’ subcommand I tried errors with:

    $ af apps
    af apps
    /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/bundler-1.1.4/lib/bundler/spec_set.rb:90:in `block in materialize:
    Could not find multi_json-1.3.6 in any of the sources (Bundler::GemNotFound)
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/bundler-1.1.4/lib/bundler/spec_set.rb:83:in `map!'
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/bundler-1.1.4/lib/bundler/spec_set.rb:83:in `materialize'
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/bundler-1.1.4/lib/bundler/definition.rb:127:in `specs'
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/bundler-1.1.4/lib/bundler/environment.rb:27:in `specs'
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/rubygems-bundler-1.0.3/lib/rubygems-bundler/noexec.rb:41: in `candidate?'
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/rubygems-bundler-1.0.3/lib/rubygems-bundler/noexec.rb:60:in `setup'
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/rubygems-bundler-1.0.3/lib/rubygems-bundler/noexec.rb:75:in `<top (required)>'
          from /Users/cnk/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:60:in `require'
          from /Users/cnk/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:60:in `rescue in require'
          from /Users/cnk/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:35:in `require'
          from /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/bin/ruby_noexec_wrapper:9:in `<main>'

I think for now I’ll ignore that problem.

Overall, this is vary clean and neat. One thing I already miss just a little is Heroku’s git-based deploy strategy. Being the version control junkie I am, there is something so right and satisfying in deploying with ‘git push heroku master’. So can you use git to deploy to AppFog? Apparently yes. It is a little more involved - you have to run a Node.js app that you can use as the url for a git post-receive hook. That let’s you automatically deploy whenever you push to the prodution (or whatever) branch.

Heroku also has some very convenient tools for moving data to and from your production database (using the taps gem). The equivalent thing in AppFog appears to involve ssh-tunnelling - see this post for details.

So far AppFog is looking promising. For most of what I do Heroke is amazingly convenient - and it now supports stacks other than Ruby on Rails. But one thing I noticed listed in the AppFog Jumpstart lists is Drupal. I am not sure if Heroku supports Drupal or not - but if I need a Drupal site and need more control than one can get hosting at Drupal Gardens, AppFog might be an option.

csrf_meta_tags

Carrying on from part1, once I had everything in place to run my tests, almost all of my functional tests failed complaining “no method csrf_meta_tags”. Ahh right, I am not running the stock Rails 2.3.5 gem in production. In February of 2011 the rails team put out a patch for a XSS vulnerability. I was concerned that some of the changes between Rails 2.3.5 and 2.3.10 might interfere with my connection to my Oracle 8i database. I was already having to do some monkey patching of the activerecord-oracle_enhanced-adapter so I could continue to use it. So instead of upgrading my gems to Rails 2.3.11, I applied the patch and then added <%= csrf_meta_tags %> to all of my layouts. Since the point of what I am doing is to upgrade my gems, for now I just commented out the offending line in each of my layouts.

SSL

OK now that I actually run some functional tests, there are a couple of other things that are not working properly. On my production (and usual development) machines, I have ssl configured and my application uses the ssl-requirement plugin to require my login pages are only accessible under ssl. I don’t want to set up SSL on this temporary location, so for now I tweaked the plugin to claim that all protocols are ‘secure’. Also during a security audit, I was asked to only pass my user cookie over https to prevent session hijacking. Since I disabled SSL, then I can never login; so I had to temporarily disable the secure cookie requirement too. Interestingly a recent Railscast discussed session hihacking and provided example code basically like what I have.

Mime-type checking

The next set of failures in my functional tests are in tests that check to see that I can only uplaod specific file types into the image and documents sections of my CMS. By default my attachement processing code, which uses attachment_fu, was using the content type provided during file upload - which largely uses the file extension to determine file type. To beef that up, I have extended attachment_fu to use the shared-mime-info ruby gem. It works fine on my production RHEL5 machine and as far as I can tell I have the same rpm + gem combination on this test box. But it isn’t working. All of my file uploads (in tests and from the browser) are getting back nil for the mime-type. Looking at the code, that implies that the library thinks it should be able to determine the file type but can’t. This may be that I am not loading the library correctly in my Gemfile: gem "shared-mime-info", :require => "shared-mime-info" But I have already had trouble getting this to work on my collegue’s Leopard Mac so there is already a work around in place for that. Again, for now I am going to make this work around trigger all the time and see what else I need to change to get this app on a supported version of Rails.

“Real” errors

So after ignoring/working around the issues above, are all my tests and cucumber features passing? Yes! I am getting one warning when I run my cucumber features. I am being told that I need to update one of my steps that is reusing other existing steps. I have been using the original syntax that uses ‘When/Given/Then’. The warning tells me to use ‘step’ instead. I made that change and then moved on.

Rails 2.3.14 on Ruby 1.8.7

So, what happens when I update to the last of the 2.3.x releases - 2.3.14 at this time. I created a new rvm gemset, updated the rails line in my Gemfile, removed my Gemfile.lock, and did a clean bundle install. When I first tried to run my app, I got complaints that I did not have Rails 2.3.5 installed - which was coming from my config/environment.rb file:

     # Specifies gem version of Rails to use when vendor/rails is not present
     RAILS_GEM_VERSION = '2.3.5' unless defined? RAILS_GEM_VERSION

So now let’s run our tests again. They pass, but this time with some additional deprecation warnings. The first set were complaints about the preferred location of the tasks directory inside my plugins. Errors like:

    DEPRECATION WARNING: Rake tasks in vendor/plugins/aftfilr/tasks are deprecated. Use lib/tasks instead.
    (called from /home/cnk/.rvm/gems/ruby-1.8.7-p352@rails2.3.14/gems/rails-2.3.14/lib/tasks/rails.rb:10)

None of the directories that rake was complaining about actually contained any tasks so the simplest option would seem to be to remove the offending directories. Or it would be if they were in code I had written. But all of those plugins were code I was including using git submodules and pointing directly to the upstream repositories. I suspect that none of those plugins have been updated in a very long time. Perhaps the best/easiest option is to just include those plugins directly in my code. Which plugins do I need to alter: aftflr, aftimagr, authorization, and mimetype_fu. Of the 4, the only one that has had any updates is mimetype_fu. Mimetype_fu is used when I have to work around the lack of shared-mime-info code on the Mac (or on this test box). For now I think I’ll just include this in my code as is.

    NOTE: Gem.source_index is deprecated, use Specification. It will be removed on or after 2011-11-01.
    Gem.source_index called from
    /home/cnk/.rvm/gems/ruby-1.8.7-p352@rails2.3.14/gems/rails-2.3.14/lib/rails/gem_dependency.rb:21.

    DEPRECATION WARNING: Kernel#returning has been deprecated in favor of Object#tap.
    (called from enable_activerecord at /home/cnk/rails/empcms2.3.5/vendor/plugins/will_paginate/lib/will_paginate.rb:39)

    DEPRECATION WARNING: Kernel#returning has been deprecated in favor of Object#tap.
    (called from generate_img_css_classes at /home/cnk/rails/empcms2.3.5/lib/aftimagr.rb:22)

Upgrading old software is often painful - but it is especially painful if you wait for a long time so that there are LOTS of changes between the old and new. The main thing that had prevented me upgrading before was a direct connection from my Rails CMS to a legacy Oracle 8i database. I no longer need to do that, so perhaps it is time to catch up with modern Rails.

There are a lot of blog posts about upgrading from Rails 2 to Rails 3; one of the most useful seems to be this one from Simon Carletti that was written before Rails 3 actually came out: The Road to Rails 3: make your Rails 2.3 project more Rails 3 oriented

His recommended first step is to get your Rails 2.3 app running using Bundler to manage your dependencies. The bundler web site has a page on how to use bundler with Rails 2.3 Seems fairly straightforward so I made the adjustments in the Rails boot files and moved my “config.gem” lines from the environment files into a Gemfile. One thing that had been kind of odd in my original project is that Shoulda, my favorite testing tool, would only seem to work if I required it globally, e.g. in config/environment.rb rather than in config/environments/test.rb. Since I am trying to change only one thing at a time, I put shoulda in the global part of my Gemfile and not just in a test section.

Which ruby?

OK so now that I have a Gemfile, which ruby should I try installing it in. Ruby 1.8.7 has had its last feature release and everyone has moved on to Ruby 1.9.3, which should be backwards compatible with Rails 2.3.5. Since that is the only ruby I currently have installed in RVM on my Lion laptop, let’s start with that one. I created a gemset (and an .rvmrc file) and then did ‘bundle install’. I had to do a little fiddling with versions, etc. Some things I knew wouldn’t work with the newer version, for example rails and shoulda. Others I didn’t know but guessed that if there had been more than a couple of minor version updates since the versions I am running in production, the new versions probably wouldn’t work with my older code. So I was fairly aggressive in specifying exactly which versions to use.

Once I had a promising looking Gemfile and installed gems, I tried running the rails console - and then one of the stock rake tasks. Sadly both failed. The script/console error is:

    (brazen:~/rails/empcms2.3.5) $ script/console
    script/console
    /Users/cnk/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in
    `require': cannot load such file -- script/../config/boot (LoadError)
            from
            /Users/cnk/.rvm/rubies/ruby-1.9.3-p194/lib/ruby/site_ruby/1.9.1/rubygems/custom_require.rb:36:in
            `require'
            from script/console:2:in `<main>'

And the rake error is:

    (brazen:~/rails/empcms2.3.5) $ rake db:schema:dump
    rake db:schema:dump
    WARNING: 'require 'rake/rdoctask'' is deprecated.  Please use 'require 'rdoc/task' (in RDoc 2.4.2+)' instead.
        at
        /Users/cnk/.rvm/gems/ruby-1.9.3-p194@global/gems/rake-0.9.2.2/lib/rake/rdoctask.rb
    rake aborted!
    undefined local variable or method `version_requirements' for
    #<Rails::GemDependency:0x007fb19601a4b0>

    Tasks: TOP => db:schema:dump => environment
    (See full trace by running task with --trace)

The rdoc warning is harmless (though annoying) - the alumni code running 2.3.14 is running fine despite similar warnings - but the others are showstopping.

There are some differences between where Ruby 1.8.7 and 1.9.3 will search for files when you ‘require’ or ‘load’ them. I am not sure if that is the cause of the problems I had, but I decided that perhaps upgrading one thing at a time might be the better part of valor. Let’s try falling back to Ruby 1.8.7.

Ruby 1.8.7

Unfortunately I can’t install ruby 1.8.7 on this Lion laptop - or at least not without a lot of work:

    $ rvm install 1.8.7
    rvm install 1.8.7
    The provided compiler '/usr/bin/gcc' is LLVM based, it is not yet
    fully supported by ruby and gems, please read `rvm re\
    quirements`.
    [11:56 PM] (brazen:~/rails/empcms2.3.5) $ rvm requirements
    rvm requirements

      Notes for Mac OS X 10.7.4, Xcode 4.3.3.

    For MacRuby: Install LLVM first.

    For JRuby:  Install the JDK. See
    http://developer.apple.com/java/download/  # Current Java version "1.6.0_26"
    For IronRuby: Install Mono >= 2.6
    For Ruby 1.9.3: Install libksba # If using Homebrew, 'brew install libksba'

    You can use & download osx-gcc-installer:
    https://github.com/kennethreitz/osx-gcc-installer ** NOTE: Currently,
    Node.js is having issues building with osx-gcc-installer. The only fix
    is to install Xcode over osx-gcc-installer.

    We had reports of http://hpc.sourceforge.net/ making things work, but
    it looks like not easiest/safest to setup.

    To use an RVM installed Ruby as default, instead of the system ruby:

        rvm install 1.8.7 # installs patch 357: closest supported version
        rvm system ; rvm gemset export system.gems ; rvm 1.8.7 ; rvm
        gemset import system.gems # migrate your gems
        rvm alias create default 1.8.7

    And reopen your terminal windows.

    Xcode 4.2:
     * is only supported by ruby 1.9.3+
     * it breaks gems with native extensions, especially DB drivers.

    Xcode 4.3+ users
    - please be warned
    - only ruby-1.9.3-p125+ is partially supported
    - in case of any compilation issues:
     * downgrade to Xcode 4.1
     * uninstall Xcode and install osx-gcc-installer
    and reinstall your rubies.

Trying to install REE gives a similar message about not liking my gcc compiler. It suggests installing “osx-gcc-installer”.

I don’t love the advice about downgrading Xcode - especially with the other note about osx-gcc-installer not playing well with Node.js. I am not currently using Node.js - but I am interested in trying it. So let’s do this on Linux.

On a RHEL5 VPS

I already have RVM and Ruby 1.8.7p352 installed on the VPS I share with friends. So I tarred up and moved the code from my Mac to that server. Altered the .rvmrc to create a gemset for ruby 1.8.7 and rails 2.3.5. I did not have bundler in that ruby so I installed it into the gemset by hand - and then did “bundle install to get the rest

    $ gem install bundler
    $ bundle install

Looks fine - but I need to configure my database. It is getting late. Take that up tomorrow.