Using IPython as your Python REPL

I had heard about IPython on my Twitter feed from people like Titus Brown but only in the context of IPython Notebooks for sharing data and calculations. I had been meaning to look into it because I figured at some point someone would want our group at work to do something about sharing how to do scientific analyses, etc. and that knowing IPython might come in handy. What I didn’t know is that you can use IPython - without the notebook component - as a replacement for the standard python REPL, kind of like how lots of people use pry instead of Ruby’s built in irb.

I wish I had looked into this sooner. IPython is more like the python REPL I needed when I was starting out in Python. I haven’t had a chance to use it a lot - for one thing I installed IPython in my virtual environment and my current environment doesn’t really have anything in it to play with. But already I am hopeful.

The main thing that makes running Python in a shell inside emacs so painful is that copying code and pasting into the emacs window makes the indenting go all weird. Can ipython fix that? I am not sure. But it does have a %paste macro. Does that help? Perhaps it helps for pasting with the paste buffer. But my usual emacs workflow of copying a chunk of code into my kill ring and then yanking it into the shell is still broken due to whitespace issues (unless of course the code is actually not indented, which rarely happens). Perhaps %run will help - at least to get some code into my REPL so I can play with it.

ipython has a nice readline history feature. One nice option is being able to get the output from the last 3 commands you ran with: _, __, and ___. I am also very impressed with being able to run system commands and assign the output to python variables, e.g. %cd /Users/me myfiles = %ls .bash*

Or something like:

    pattern = "imp"
    !grep -rF $pattern ipython/*

To get specific information on an object, you can use the magic commands %pdoc, %pdef, %psource and ```%pfile`` OMG! This is like pry’s cd into an object and look around. Instructions for using these magic commangs as part of some other python program are in the docs. This seems to me to be a lot like binding.pry to start a REPL somehwhere.

Since %automagic is turned ON by default, you can often just do ls or cd without the leading %. You can toggle the automagic by typing %automagic until you get what you want.

To see what things are included in automagic: %magic -brief

There is also a set of docs on how to use ipython as a systems shell: https://ipython.org/ipython-doc/dev/interactive/shell.html

Reloading files inside the REPL

The IPython.lib.deepreload module allows you to recursively reload a module: changes made to any of its dependencies will be reloaded without having to exit. To start using it, do:

    from IPython.lib.deepreload import reload as dreload

Misc

Automatic quoting

The automatic quoting sounded useful but when I played around with it, it was odd. All I really want is an equivalent of Ruby’s %w(foo bar baz) so I don’t have to type a boatload of “ and , to get ["foo", "bar", "baz"]. Unfortunately this is not that feature.

Spying on IPython

Or “How IPython keeps track of what you have running where”. When the ipython notebook started up, it created the nbserver-###.json file. When I started actually running code in an .ipynb file, then it needed a running kernel - so it spun one up and created the kernel=###.json file

    /Users/cnk/.ipython/profile_default/security:
      total used in directory 64 available 230555563
      drwx------   8 cnk  staff    272 May 14 18:26 .
      drwxr-xr-x  10 cnk  staff    340 May 14 18:26 ..
      -rw-rw-r--   1 cnk  staff    245 May 14 18:26 kernel-fc238266-ac92-423c-99d0-c6faeb911a15.json
      -rw-rw-r--   1 cnk  staff    187 May 14 18:26 nbserver-61240.json

This part of the docs has lots of useful information about how ipython processes communicate with the kernel - message passing, security (only bind to localhost OR use ssh tunnels to encrypt trafic and keep it private).

Installing IPython

For our new project we are trying to use Python 3, so I want to evaluate IPython on the latest version of Python, which currently is 3.4.3. I am using pyenv and pyvirtualenv to manage my python installs. So with those all set up, I installed python 3.4.3 and created a virtualenv to use for ipython.

    pyenv install 3.4.3
    pyenv virtualenv 3.4.3 ipython-3.4.3
    mkdir ~/Code/ipython
    cd ~/Code/ipython
    echo 'ipython-3.4.3' > .python-version
    pip install --upgrade ipython
    # the pip install above gave me messages about upgrading pip, so I did:
    pip install --upgrade pip

I haven’t used IPython before - or should I say Jupyter since that is the project’s new name. So I watched a couple of the introductory videos I found on YouTube. This video told me I needed to add the python 3 kernel to jupyter, so I tried to do that but kept getting messages about missing pieces every time I tried to run ipython.

    $ ipython3 kernelspec install-self
    # error message about not having pyzmq
    $ pip install pyzmq
    $ ipython3 kernelspec install-self

    $ pip freeze
    gnureadline==6.3.3
    ipython==3.1.0
    pyzmq==14.6.0

    $ ipython
    ImportError: No module named 'jinja2'
    $ pip install jinja2
    ImportError: No module named 'tornado'
    $ pip install tornado

    $ ipython
    IPython notebook format depends on the jsonschema package:
    $ pip install jsonschema

    $ pip freeze
    certifi==2015.4.28
    gnureadline==6.3.3
    ipython==3.1.0
    Jinja2==2.7.3
    jsonschema==2.4.0
    MarkupSafe==0.23
    pyzmq==14.6.0
    tornado==4.1

What I had overlooked was an install target that would install not only ipython but all the things it typically depends on / uses. Once I installed ‘all’ I got a usable ipython.

    $ pip install ipython[all]
    $ pip freeze
    alabaster==0.7.4
    Babel==1.3
    certifi==2015.4.28
    docutils==0.12
    gnureadline==6.3.3
    ipython==3.1.0
    Jinja2==2.7.3
    jsonschema==2.4.0
    MarkupSafe==0.23
    mistune==0.5.1
    nose==1.3.6
    numpydoc==0.5
    ptyprocess==0.4
    Pygments==2.0.2
    pytz==2015.2
    pyzmq==14.6.0
    requests==2.7.0
    six==1.9.0
    snowballstemmer==1.2.0
    Sphinx==1.3.1
    sphinx-rtd-theme==0.1.8
    terminado==0.5
    tornado==4.1

Exporting from IPython to other formats

One of the things we are most interested in for work is the export formats that IPython / Jupyter supports so I tried out all the export formats. The PDF renderer was looking for pandoc. According to the pandoc install instructions, before I can install pandoc, I need a version of LaTeX for the Mac. So, I installed MacTeX from their MacTeX.pkg. And then I installed pandoc with homebrew: brew install pandoc. I still can’t export PDFs from the drop down menu in the iPython web interface:

    nbconvert failed: command could not be found pdflatex

However, from the command line, I can convert my test document to latex:

    $ ipython nbconvert --to latex --template article CNKTest.ipynb

produces the file CNKTest.tex which I can open and view in Latexit (which came from MacTeX).

Matplotlib

Some of the things I am playing aroud with want matplotlib - which does not appear to be installed in my virtualenv. So I ran pip install matplotlib which says it did:

    Installing collected packages: python-dateutil, numpy, pyparsing, matplotlib
    Successfully installed matplotlib-1.4.3 numpy-1.9.2 pyparsing-2.0.3 python-dateutil-2.4.2

And now I can load matplotlib inside a running ipython session with %matplotlib (or I could load it when I start the session using the command line argument: ipython --matplotlib

Discourse installed with Vagrant and Docker

I guess I am old school but I prefer mailing lists to web forums - probably because I vastly prefer mutt to any forum web site I have seen. But if I have to use a discussion board, then by far the best one I have seen is Discourse. I first heard about it in a couple of episodes of the Ruby Rogues podcast.1 2 When the Rogues moved their mailing list from Google Groups to Discourse, I was a little annoyed because Discourse didn’t support ‘just email it all to me’ and I just never get around to checking web sites. Fortunately the good folks at Discourse added setting that allow folks like me to get all our Ruby Parley goodness by email. But for those who like web forums, Discourse has some really nice features - keeping track of what is new, likes, bookmarks, follow individual posts, private messages and a really nice reputation system.

At work we are looking for a discussion board for a project we are doing and I suggested Discourse. In addition to the hosted discussions, there is an open source version of the same code available on GitHub. The recommended way to install Discourse is to use the supplied Docker images.

Setting up my Vagrant

To get started I needed an Ubuntu 14 LTS disk image. The one distributed by Hashicorp labeled “lattice/ubuntu-trusty-64” is configured to use 4G memory and isn’t running chef or puppet (so nothing to interfer with the Docker stuff from Discourse).

    vagrant init lattice/ubuntu-trusty-64
    vagrant up
    # first thing it did was download the vbox image
    vagrant ssh
    wget -qO- https://get.docker.com/ | sh

    ... lots of output about updating packages and installing docker

    If you would like to use Docker as a non-root user, you should now consider
    adding your user to the "docker" group with something like:

      sudo usermod -aG docker vagrant

Installing Discourse

    mkdir /var/discourse
    git clone https://github.com/discourse/discourse_docker.git /var/discourse
    cd /var/discourse
    cp samples/standalone.yml containers/app.yml
    # edited email settings in app.yml per instructions

I misread the instructions and instead of typing ./launcher bootstrap app, I did ./launcher start app. That gave some reasonable looking output before it bailed out complaining:

    Unable to find image 'local_discourse/app:latest' locally
    Pulling repository local_discourse/app
    FATA[0002] Error: image local_discourse/app:latest not found

Doing ./launcher rebuild app did reasonable looking things like starting databases, checking out code from git, running database migrations, and compiling assets.

I tried modifying my /etc/hosts file to include discourse.example.com but no dice - I think because my Vagrant isn’t exposing port 80. So I shut down the vagrant box and edited the Vagrant file to uncomment

    config.vm.network "forwarded_port", guest: 80, host: 8080

Now http://discourse.example.com:8080 works. Since I can don’t really need this to answer to a specific name, I took the modification back out of /etc/hosts and will just use http://localhost:8080 to play with configuring and testing Discourse.

Rails and JavaScript: JQuery, JSON, Oh My!

With Rails ‘respond_to’ method, it is really easy to return differnt kinds of data from the same controller method based on the url extension. I was really happy with this when I was building a a site to provide a read-only version of an event calendar I had built years earlier on the ArsDigita Community System. To get data out of that Oracle database, I built a Rails web site that let users ask for events by category, sponsor, or lecture series with a variety of date restrictions. The data was consumed by several different clients - two who wanted XML and several others who wanted JSON. I could use exactly the same URLs by just adding .xml or .json to the url. Even better, I could have controller render an html version of the same data which developers could use to preview the result sets. Once they had all the parameters the way they wanted, they just copied the url and added ‘.json’ ahd used it in their web pages.

But now I actually want to move into the 21st century and build some web pages with AJAX interactions. Rails has supported a variety of methods of working with Javascript starting with the RJS methods in the version 1.x days. Currently Rails (4.2.0) uses JQuery by default - including a jquery-ujs package which, when combined with the ‘remote: true’ argument to ‘form_for’, makes it simple to have your forms submit via AJAX instead of a normal server request.

First, the regular form_for helper:

    # This rails helper
    <%= form_for(@course) do |f| %>

    # Produces this html
    <form class="new_course" id="new_course" action="/courses" accept-charset="UTF-8" method="post">
      <input name="utf8" type="hidden" value="✓">
      <input type="hidden" name="authenticity_token" value="YE006c08mOnIYENufqJJZVrV3yAo1sHOgk9EJ9ZqigLIUktT6Vg0Px8FMx5GTl54ewIKYOaWqJYpWgiPEpQDZQ==">

But with ‘remote: true’ (note no authenticity_token):

    # This rails helper
    <%= form_for(@course, remote: true) do |f| %>

    # Produces this html
    <form class="new_course" id="new_course" action="/courses" accept-charset="UTF-8" data-remote="true" method="post">
      <input name="utf8" type="hidden" value="✓">

This comes into the server and is interpretted as a JS request:

    Started POST "/courses" for <ip> at 2015-03-12 23:07:10 -0700
    Processing by Empcms4Tutors::CoursesController#create as JS

which will fall into the format.js case in the normal respond_to stanza, which by default, will try to render the create.js.erb template in your courses views. This view should jquery directives that will be executed in the browser.

    $('#all_courses').append("<%= escape_javascript(render partial: 'table_row', locals: { course: @course }) %>");
    $('#course_name').val(''); // clear form input field
    $('#error_explanation').html(''); // clear error message

This flow has some pros and cons. On the bright side, it is relatively easy (as long as you remember to use ‘escape_javascript’ on any html you produce. But it means you have a fair amount of javascript scattered small files in your views directory. It is JS that manipulates your html so in that sense it is a view. But even if I am less than enthusiastic about that aspect, the big selling point is that I can reuse the code in my view partials. The exact same embedded ruby (erb) that created the table rows that loaded when the page first loaded is the erb that creates the new row JQuery inserts when I finish adding a new course.

Speaking JSON

If you want to keep all of your JavaScript together, for example if you are using a more extensive front end framework, then you may want to have your AJAX interactions communicate as JSON. The adjustments you need to make for that start with adding a data-type argument to your form tag:

    # This rails helper
    <%= form_for(@course, remote: true, format: json) do |f| %>

    # Produces this html
    <form class="new_course" id="new_course" data-type="json" action="/courses" accept-charset="UTF-8" data-remote="true" method="post">
      <input name="utf8" type="hidden" value="✓">

    # Which shows up as this on the server
    Processing by Empcms4Tutors::CoursesController#create as JSON

If you don’t add a different format section, this will also end up in the format.js section and will send back the jquery selectors and manipulations as before - with a Content-Type header of ‘text/javascript’. However, since the browser wasn’t expecting to get JavaScript, it doesn’t evaluate it (at least Chrome doesn’t). So I can see the server responding without error, and the data coming into the browser with a status code of 200. No JavaScript errors register in the console - but nothing happens on the page. All in all my least favorite type of error - no error, just nothing happens.

So to correct that, we add a ‘format.json’ section to our ‘respond_to’ block:

    format.json { render json: @course }

So now the browser gets back a JSON representation of the new item with a Content-Type header of ‘application/json’. Then it is up to us to write the client-side JavaScript - for example to add the table row with the new element. That might look something like this (untested code):

    $('#all_courses).on('ajax:success', function(event, data, status, xhr) {
                                        // add the table row with the course name, links, etc.
                    }.on('ajax:error', function(event, xhr, status,error) {
                                       // insert the failure message near the form
                                       $('form').append(xhr.responseText)
                    });

This keeps all your JavaScript on the client side - but at the expense of mixing some html into it as you build the new table row.

Upgrading my laptop to Yosemite

We just got new Macs at work - which came with Yosemite preinstalled. There are a few odds and ends that I am not thrilled with, but overall I have found the transition from Lion to be quite smooth. So guess it is time to upgrade my personal laptop - before any more security issues are announced.

I went to the App Store and downloaded the installer and ran it. All seemed to be going well until the point where the installer thought it had “2 minutes left”. Two hours later, I gave up and went to bed, leaving it running. I have no idea when it actually finished but in the morning, my machine was asleep. When I woke it up, I got the odd gray screen overlay, with the blinking gray startup bars near the bottom - like you get when you have run the battery completely out. I was wondering if either the install had failed (probably not, since the screen behind the gray overlay was the new, flat, Yosemite look) or if the machine had overheated and crashed. It got very hot during the install but I had not heard the fan running. I did what I could to leave the bottom of the machine with good airflow so it could cool off but that was all I could do for that issue.

Anyway, despite the odd gray screen, the machine started up OK, let me log in, and then ran some “setting up your new mac” things. Looks OK. I launched the app store and installed the two security updates that have come out since Yosemite launched. I also downloaded the Mac office apps, pages, keynote, numbers….

XCode

I downloaded the newest XCode (6.1) and installed it. The first time it launched, I was asked to allow it to upgrade the device support - which it did and then promptly crashed. When I tried to launch it again, it crashed again. But third time is the charm. It again asked if it could run some (different) updates, but this time, it then gave me the normal XCcode interface. Until I get around to learning iOS programming, I don’t really have much use for the XCode GUI - except for using the iOS simulator as one of the targets for Karma to run JavaScript tests in a browser. The reason I need XCode is for the compiler. To get the command line tools installed, I ran xcode-select --install and agreed to install the command line tools. Then I ran xcodebuild --license. This was supposedly to accept the Xcode license - but just like when I did it for setting up my work machine, I got xcodebuild: error: invalid option '--license'. That Chris said that indicated that I had already accepted the license; not sure about any of that, but it doesn’t seem to have interfered with compiling stuff at work, so I am not going to worry about it.

X11

X11 no longer comes with OsX, but when I tried to use the X11 icon in my dock, it helpfully redirected me to this page which allowed my to download XQuartz 2.7.7 The installer ran just fine and at the end told me I had to log out and back in again to make this the default X11 for my machine.

Homebrew

I should have done a little checking and preparation before I did my upgrade. At least with Mac Ports, you generally can’t run any of the port commands after you have upgraded the OS. This means that it is really hard to figure out which packages you had installed (in particular, which were installed explicitly and which were installed because they were dependencies for something else you asked for) unless you ask for a list before doing the upgrade. While I was looking for blog posts about updating my homebrew-installed software, I found this blog post which explains why my Yosemite upgrade took FOREVER once it got to the 2 minutes left stages. Something about moving /usr/local and then moving all the files back… one by one. Oh well, at least now I know what that was about.

Fortunately, it looks like Homebrew will still basically work after the upgrade. I can run brew list and get a reasonable list of installed packages: emacs, docbook, graphviz, imagemagick, mongodb, mysql, node, postgresql, python, wget, etc. The blog post above recommended running brew doctor which gave me several sets of advice:

• some broken symlinks to remove with brew prune
• warnings about unlinked kegs
• a keg-only formula for libxml2 that is linked into /usr/local
• a warning that I should run brew update to get the latest formula info

The last one seemed reasonable, so I started there. The output of that command (and some reading of docs and the install script) explained why brew survived my upgrade. Homebrew just uses the system ruby - which is always there and should always run on the installed OS. And it explains why I don’t see separate ‘upgrade homebrew itself’ and ‘update the package list’ commands. Both are taken care of by running brew update which checks out the latest software and package information from Github:

    $ brew update
    Checking out files: 100% (1668/1668), done.
    Updated Homebrew from ef6d160e to d9a69a40.
    ==> New Formulae
    ==> Updated Formulae
    ==> Deleted Formulae

The first piece of advice also seemed reasonable - clean up dangling symlinks:

    $ brew prune
    Pruned 0 dead formulae
    Tapped 36 formulae
    Pruned 30 symbolic links from /usr/local

Then, I ran brew outdated to see what needs updating:

    $ brew outdated
    cairo (1.12.16_1 < 1.14.0)
    cmake (2.8.8, 2.8.12.2, 3.0.2 < 3.1.0)
    emacs (24.1, 24.3 < 24.4)
    ffmpeg (1.2.1, 2.2.3, 2.3.3 < 2.5.2)
    freetype (2.5.3_1 < 2.5.5)
    gettext (0.18.3.2, 0.19.2 < 0.19.4)
    git (1.8.0.2, 1.8.1.4, 2.0.0, 2.1.1 < 2.2.1)
    glib (2.42.0 < 2.42.1)
    gtk+ (2.24.24 < 2.24.25)
    harfbuzz (0.9.35 < 0.9.37)
    icu4c (53.1 < 54.1)
    imagemagick (6.8.0-10, 6.8.9-1, 6.8.9-7 < 6.8.9-8)
    libevent (2.0.21 < 2.0.21_1)
    libgpg-error (1.10, 1.13, 1.16 < 1.17)
    libksba (1.3.0, 1.3.1 < 1.3.2)
    libpng (1.6.10, 1.6.13 < 1.6.16)
    librsvg (2.36.3 < 2.36.3_1)
    libtool (2.4.2 < 2.4.4)
    libxml2 (2.9.1 < 2.9.2)
    mongodb (2.6.1, 2.6.4_1 < 2.6.6)
    mpfr (3.1.2, 3.1.2-p8 < 3.1.2-p10)
    mysql (5.5.25, 5.6.19, 5.6.21 < 5.6.22)
    node (0.10.9, 0.10.28 < 0.10.35)
    openssl (1.0.1e, 1.0.1h, 1.0.1i < 1.0.1j_1)
    postgresql (9.1.4, 9.2.4, 9.3.4, 9.3.5_1 < 9.4.0)
    python (2.7.5, 2.7.7_1, 2.7.8_1 < 2.7.9)
    redis (2.8.9, 2.8.10, 2.8.17 < 2.8.19)
    sqlite (3.7.15.2, 3.7.17, 3.8.4.3, 3.8.6 < 3.8.7.4)
    webp (0.4.0_1 < 0.4.2_1)
    wget (1.13.4, 1.15_1, 1.15_2 < 1.16.1)
    xvid (1.3.2 < 1.3.3)

I think I want to update just about everything. But I am a little concerned about upgrading the databases. Let’s start by pinning them, then upgrading everything else, and then upgrading them one at a time. But first, do they run now?

Looks like mysql is running - I see processes for mysqld_safe and mysqld in ps and I can log in as root and poke around. The postgresql server is not running at the moment and I mostly have not been using it. So I am just going to pin those two packages and upgrade everything else:

    $ brew pin mysql
    $ brew pin posgresql
    $ brew upgrade

    ==> Upgrading 29 outdated packages, with result:
    cairo 1.14.0, cmake 3.1.0, emacs 24.4, ffmpeg 2.5.2, freetype 2.5.5,
    gettext 0.19.4, git 2.2.1, glib 2.42.1, gtk+ 2.24.25, harfbuzz 0.9.37,
    icu4c 54.1, imagemagick 6.8.9-8, libevent 2.0.21_1, libgpg-error 1.17,
    libksba 1.3.2, libpng 1.6.16, librsvg 2.36.3_1, libtool 2.4.4, libxml2
    2.9.2, mongodb 2.6.6, mpfr 3.1.2-p10, node 0.10.35, openssl 1.0.1j_1,
    python 2.7.9, redis 2.8.19, sqlite 3.8.7.4, webp 0.4.2_1, wget 1.16.1,
    xvid 1.3.3

    ==> Not upgrading 2 pinned packages:
    mysql 5.6.22, postgresql 9.4.0

    Error: You must `brew link fontconfig pixman' before cairo can be installed

Then the updates all ran - some giving messages, mostly about how to launch daemons or warning you not to link certain libraries because they would conflict with the OsX versions. To pick up the last few upgrades, I did as suggested and ran brew link fontconfig pixman and then reran brew upgrade. I will save the output of all of these upgrades in case I need to go back to it.

MySQL

The MySQL upgrade is only one minor version different, so I think I’ll try to do that upgrade. About the only database I care about in the current one might be the cardsharp_dev database, so I am not too worried about the data even if there is some strange incompatibility.

    $ launchctl unload ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
    $ brew unpin mysql
    $ brew update mysql
    $ launchctl load ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist

Logged in and looked around. Seems fine at least at a glance.

Starting and stoping services

While I was looking around for how to stop MySQL before the upgrade, I found two useful resources, this launchctl tutorial and this blog post on a brew command to replace all this: brew services

    $ brew tap homebrew/boneyard
    brew tap homebrew/boneyard
    Cloning into '/usr/local/Library/Taps/homebrew/homebrew-boneyard'...
    remote: Counting objects: 714, done.
    remote: Compressing objects: 100% (4/4), done.
    remote: Total 714 (delta 1), reused 0 (delta 0)
    Receiving objects: 100% (714/714), 171.26 KiB | 0 bytes/s, done.
    Resolving deltas: 100% (335/335), done.
    Checking connectivity... done.
    Tapped 65 formulae

    [08:51 PM] (brazen:~) $ brew services stop mysql
    brew services stop mysql
    Warning: brew services is unsupported and will be removed soon.
    You should use launchctl instead.
    Please feel free volunteer to support it in a tap.

    Stopping `mysql`... (might take a while)
    ==> Successfully stopped `mysql` (label: homebrew.mxcl.mysql)

    [08:52 PM] (brazen:~) $ brew services start mysql
    brew services start mysql
    Warning: brew services is unsupported and will be removed soon.
    You should use launchctl instead.
    Please feel free volunteer to support it in a tap.

    ==> Successfully started `mysql` (label: homebrew.mxcl.mysql)

Postgres

Perpahs I should leave it but I would really like to have everything upgraded for the start of the new year. I had trouble getting to old databases after a previous postgres upgrade. So first I want to take a backup. And to do that, I need to start the database server…..

    $ postgres -D /usr/local/var/postgres
    postgres -D /usr/local/var/postgres
    FATAL:  could not open directory "pg_tblspc": No such file or directory

It looks like the [Yosemite upgrade removed a few empty directories] (http://stackoverflow.com/questions/25970132/pg-tblspc-missing-after-installation-of-os-x-yosemite) that we need:

    $ mkdir /usr/local/var/postgres/pg_tblspc
    $ mkdir /usr/local/var/postgres/pg_twophase
    $ mkdir /usr/local/var/postgres/pg_stat_tmp
    $ postgres -D /usr/local/var/postgres
      LOG:  database system was shut down at 2014-10-02 22:09:43 PDT
      LOG:  creating missing WAL directory "pg_xlog/archive_status"
      LOG:  database system is ready to accept connections
      LOG:  autovacuum launcher started

I logged in and looked around. Looks like there are 2 tablespaces (\db) both owned by me - and a single schema (\dn) - also owned by me. What I missed, was how to list the databases: \l. So I missed doing the pg_dump before I upgraded the database. Fortunately there is an upgrade command which I found detailed at https://kkob.us/2014/12/20/homebrew-and-postgresql-9-4/.

    $ mv /usr/local/var/postgres /usr/local/var/postgres9.3
    $ brew unpin postgresql
    $ brew upgrade postgresql
    $ initdb /usr/local/var/postgres -E utf8
    $ pg_ctl -D /usr/local/var/postgres -l logfile start
    server starting
    $ psql
    psql: FATAL:  database "cnk" does not exist
    $ psql
    psql (9.4.0)
    cnk=#

    $ pg_ctl stop
    waiting for server to shut down.... done
    server stopped

    $ pg_upgrade -d /usr/local/var/postgres9.3 -D /usr/local/var/postgres \
     -b /usr/local/Cellar/postgresql/9.3.5_1/bin/ -B /usr/local/Cellar/postgresql/9.4.0/bin/ -v

    ... This did some checks then dumped the old databases:
        alensonline
        cms_dev
        cms_test
        cnk
        marysplace_dev
        marysplace_test
        postgres
        rails_rumble_13_development
        template1

There was a lot more output - and eventually the upgrade failed because I had already created the database ‘cnk’. So let’s remove the new stuff and try again:

    $ rm -rf /usr/local/var/postgres
    $ rm /usr/local/var/pg_upgrade_*
    $ initdb /usr/local/var/postgres -E utf8
    $ pg_upgrade -d /usr/local/var/postgres9.3 -D /usr/local/var/postgres \
     -b /usr/local/Cellar/postgresql/9.3.5_1/bin/ -B /usr/local/Cellar/postgresql/9.4.0/bin/ -v
    ......
    Upgrade Complete
    ----------------
    Optimizer statistics are not transferred by pg_upgrade so,
    once you start the new server, consider running:
        analyze_new_cluster.sh

    Running this script will delete the old cluster's data files:
        delete_old_cluster.sh

The end of the initdb output mentioned it was enabling ‘trust authentication’ for all local connections. I only want my user to be able to log into postgres. So before starting up the database, I edited /usr/local/var/postgres/pg_hba.conf to change the user from ‘all’ to ‘cnk’.

    $ pg_ctl -D /usr/local/var/postgres -l /usr/local/var/postgres/logfile start
    $ psql
    psql (9.4.0)
    Type "help" for help.

    cnk=# \l
                                           List of databases
                Name             | Owner | Encoding |   Collate   |    Ctype    | Access privileges
    -----------------------------+-------+----------+-------------+-------------+-------------------
     alensonline                 | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     cms_dev                     | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     cms_test                    | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     cnk                         | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     marysplace_dev              | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     marysplace_test             | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     postgres                    | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     rails_rumble_13_development | Mac   | UTF8     | en_US.UTF-8 | en_US.UTF-8 |
     template0                   | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 | =c/cnk           +
                                 |       |          |             |             | cnk=CTc/cnk
     template1                   | cnk   | UTF8     | en_US.UTF-8 | en_US.UTF-8 | cnk=CTc/cnk      +
                                 |       |          |             |             | =c/cnk
    (10 rows)

    cnk=# \c alensonline
    You are now connected to database "alensonline" as user "cnk".
    alensonline=# \d
                      List of relations
     Schema |          Name           |   Type   | Owner
    --------+-------------------------+----------+-------
     public | assets                  | table    | cnk
     public | assets_id_seq           | sequence | cnk

Looks like everything made it.