Tonight I was debugging something for a client. Our Rails app uses SyslogLogger to log to syslog (instead of the files in log/*.log, like Rails is set up by default. We think there are some wins using syslog).

I was trying to debug why some logging statements would show up, and some didn’t. In fact, logger.info and logger.debug weren’t working for us. These correspond, by default, to syslog log levels info and debug (oddly enough).

This blog entry will explain how what settings you need to tweak to have Console.app display things at a higher log level. To see log level debug in Console.app. (Yes, this is the part of the article where I input search terms I used in trying to find this information, so some other sod can find it too.)

Syslog is one of those unix components that Apple pulled the guts out of and left a small shell that’s compatible with the rest of Unix. Domain of the Bored has an excellent series of articles on ASL. There’s also some really good documentation from an (I’m guessing) Apple engineer on the darwin-dev mailing list on ASL.

So great. ASL controls syslog. If ASL says something isn’t stored, syslog doesn’t ever see it – even if it was a log level syslog would normally care about.

We need to add an entry to asl.conf. man asl.conf gives us syntax, etc. All we want to do is capture all log levels for our app, rails_app_mine.

edit /etc/asl.conf with your favorite editor. Add the following line:

? [<= Level debug] [= Sender rails_app_mine] store
Sender is the syslog process. We set this in our Rails app by creating the SyslogLogger object like so:

config.logger = RAILS_DEFAULT_LOGGER = SyslogLogger.new("rails_app_mine")

Now syslog will see your lines, and you can configure syslog using normal syslog mechanisms (like sending your rails_app_mine messages to a syslog server, or syslog-ng, let’s say).

Alternatively, in /etc/asl.conf you could modify the line

? [<= Level notice] store

to instead read

? [<= Level debug] store

but that would log ALL debug statements, including ones that we really don’t care about. And seriously, there’s too much spew in Console.app already.

Make this change and reload syslog by having launchd reload it (no, -HUP-ing it doesn’t work):


sudo launchctl unload /System/Library/LaunchDaemons/com.apple.syslogd.plist; \
sleep 1;\
sudo launchctl load /System/Library/LaunchDaemons/com.apple.syslogd.plist


And you’re done.

This is a test to make sure I have the blog up and working

The Intro, and the frustration

So SVN’s command line interface could be a lot better. Specifically, svn diff.

Use case: I update and see a change someone else has made. I want to see what modifications they made relative to what it used to be… how do I do this?

The answer using the straight svn command line is so:

1. Figure out what revision the file changed under (mindful that the revision subversion reported to you when you did an update may actually not be the revision of the last change, if you have multiple projects in the same repository and some other part of the revision changed between this change and the HEAD of the repository).
2. Figure out what the revision-before-that-was. If you’ve figured out #1 by doing an svn log, well you have this number too.
3. svn diff -r revision-from-step-2:revision-from-step-1 somefile

Which sucks. For one it’s too many steps, and then there’s general svn unhelpfulness.

To make your life more ‘wonderful’, svn diff will return nothing in the following situations:

1. The file has not changed between the revisions you specified. (Obviously you didn’t do steps #1 and #2, but svn is not going to tell you this)
2. The file doesn’t exist in the repository at all. (Obviously you screwed up the file name, but svn is again going to remain silent).

So if svn returns nothing you have one of these two things going on. Good luck debugging, buddy.

But yes, the suck. Non obvious, and WAY to much work if I just want to see what changes some coworker made to my codebase. This needs to be a fast step: I just want to do a code review quickly and get back to work!!!

A tool to make it better

So I wrote a tool to make it better. Granted, an inefficient shell script to make it better, but hey.

This shell script accomplishes two thing:

• Gets rid of having to do steps #1 and #2. Instead you give the shell script what i call a revision offset. (”Compare the current version to the version one change ago”). This is NOT revision number — remember that there may have been other changes since that file changed… but “revision offset” is a relative number — “N changes ago”.

  Sadly, not in English like that, but it’s a bash script — maybe I’ll rewrite it someday and use Python and PythonMethodMatcher.

• HELPS you debug. “Hey, buddy, I think that file doesn’t exist. You sure about that?”
• This tool will also helpfully tell you what it’s doing, so it’s not opaque magic — if you want to do something different you can take what the script generates, modify it and off you go. This tool is transparent and wants to help you.

The shell script is far from perfect (it’s pretty inefficient, because I’m not a master bash coder), but it does make this less frustrating.

Example Use Case

$ ls
README.txt somefile.sh someotherfile.sh
$ svn log README.txt
------------------------------------------------------------------------
# r170 | someone | 2009-04-09 22:52:59 -0400 (Thu, 09 Apr 2009) | 1 line
#
# a change
#------------------------------------------------------------------------
# r167 | someone | 2009-04-06 09:43:25 -0400 (Mon, 06 Apr 2009) | 1 line
#
# another change
#
....

The Script

Download the script from my site!

I’ve been doing a lot of Ruby on Rails Cucumber testing, and was inspired by the simplicity of controlling the flow of executing by matching on regular expressions.

The idea behind cucumber is you write English steps and assertions to test a web app. Plain English means just that: no “tell box 1 of window 1″ almost-english-but-not-quite, and not nerd English like Rails makes popular “1.days”.

Plain English.

Here’s a (slightly modified) example from something I’ve been working on:


Given a logged in, registered user
And I have a blog post that I can edit
Then I should see "Edit"


This all happens by the parser matching those phrases with a bunch of regular expression code blocks elsewhere in the app, then executing one if one matches. Which is brilliant in its simplicity.

So I wrote this same kind of engine in Python. Just a generic regex matches calling functions, suitable for any purpose (not just test driven development like Cucumber).

I put it on on GitHub. Find it at PyMethodMatch’s home on GitHub

Today I’m proud to announce a product launch: Experienced Man’s Guide To Cross-Platform Programming with wxWidgets.

When I buy a computer book I write notes in it. Sometimes these notes are back and forward references, sometimes I found a bug in something and I write the solution down, sometimes something behaves one way one place and another way another. Maybe time passes, things change, but the book has (now) outdated information.

So I write notes in the margin. When there isn’t any room in the margin I take out half a piece of paper and write my notes there.

Over the course of a few years my books become pretty note-ridden, especially if I was using the book every day.

I bought Cross-Platform GUI Programming with wxWidgets about 3 years ago, and during those 3 years I’ve been using wxWidgets almost non-stop stop clients. I have a lot of notes in this book.

So I copied all my notes from my book and put them in a handy dandy PDF, which I’m now selling as a DRM free PDF.

Since this is a PDF, and not a traditional printed book, I’m selling it like software: Version 1.0 is out today, but there will be upgrades and fixes and revision level releases so on and so forth. If you have knowledge to add, or ideas or comments, please contact me and we’ll see what we can do to make you happy.

So please support this product, and my effort to keep the wxWidgets book up to date, relevant and useful!

So, after trying to install ReviewBoard on a virtual machine (and failing), I decided to install it on my real server, a machine running Debian PPC (sarge).

Eventually I got it up and running, but here are my notes:

I decided to run it off fastcgi (asking apt-get to install lighttpd pulled in tons of other software I didn’t want… and I didn’t want to screw around with setting up mod_python – although in retrospect maybe that was silly). You need to apt-get libapache2-mod-fastcgi. EXCEPT this doesn’t work by default: you need to set your /etc/apt/sources.list file to download “non-free” software too. So you lines will look something like deb http://debian.lcs.mit.edu/debian/ testing main non-free. What makes this confusing is that there is libapache2-mod-fastcgid, which is a different implementation of the fastcgi protocol… but not what you want at all. Anyway there’s more yack about sources.list here.

Also, if you’re having Apache2 listen on an oddball port for requests to your ReviewBoard, make sure your firewall lets those ports through :).

Make sure to uncomment the line in apache2.conf that Includes the files in sites-enabled!

I thought Everburning: Fighting Review Board was going to be helpful, but turns out it mostly works out of the box (without the stuff he does).

You might have to fiddle with settings_local.py’s FORCE_SCRIPT_NAME, and what the rewrite rule in the Review Board generated Apache2 config file says, to make them agree. This will be pretty obvious when the front review board page is ugly (because the CSS didn’t load), OR you’ll get an error about URLconf defined in djblets.util.rooturl, Django tried these. URL patterns...

I think that’s it. I’m pretty sure I easy_install ReviewBoard to get it installed, but I think there might be an apt-get package for it too.

So it wasn’t a bad experience, just one I was hoping to avoid with a virtual machine for it.. oh well I guess. Hope this helps someone else.

Today I got distracted by Review Board, a tool to help companies do code reviews. Because I was too lazy to deal with the dependancies I decided to look for a virtual machine for Review Board, all configured for me.

This was probably a stupid idea in retrospect.

I found a virtual image provided by rPath/rBuilder with (come to find out) no documentation. None at all. Which is annoying because to set up Review Board you need a “superuser” for the web app.

So, in order to set up Review Board on this box you need to do these things:

1. Log into the machine. root with no password will log you into the shell
2. rm /srv/reviewboard/data/reviewboard.sqlite
3. /srv/reviewboard/manage.py syncdb. It’ll ask you if you want to create a superuser. Say yes.
4. chmod 777 /srv/reviewboard/data/reviewboard.sqlite

Yes, the chmod 777 is a security risk. However, if you were to actually deploy this you’d want to use MySQL or something like that.

The settings files are in the /srv/reviewboard/ directory.

Also: the virtual machine hosts Review Board on port 80.

What is neat about the rPath/rBuilder virtual appliance is that they offer an EC2 virtual machine with Review Board too…

Hope this saves someone some time…

Update: And, having done all that, the machine doesn’t have PySVN on it, nor developer tools (gcc, easy_install) to bootstrap it myself. Arg.

So Rails has a feature called “migrations”. The idea here is that your database will change over time and you need a way to change this incrementally as the project changes and new requirements come up. So essentially your database moves through time.

Rails also has a feature where you can define validations for a field at the model level (aka: application level, vs database level, validations): make sure it is defined, is in a certain range, etc. These are declarative statements, meaning that they are set up when the class is defined and ran when an instance is created.

In certain situations, however, there comes an issue with this. For example, if you’re using Cruise Control to do integration tests, you might find that you’re migrating down to the first migration and then migrating all the way back up to current. Which is fine and a good practice, except for one thing: your model validations don’t move through time with your migrations. This is no problem… unless you’re trying to create or modify records using these model objects in your migrations. Then you have a paradox: the model tries to validate a field, but was given no value because that field doesn’t exist yet.

Lets say in migration 2 we are creating vendors and populating some default ones:

class CreateVendors < ActiveRecord::Migration
def self.up
create_table :vendors do |t|
t.string :name
t.string :email
end

def self.down
drop_table :vendors
end
end


But we”ve been developing the app for some time, and in migration 20 we add a rating field to the Vendor table, as well as the following validation to the model:

validates_presence_of :rating, :on => :create, :message => "can't be blank"

Now when we migrate down to 0 and migrate back up, we get an error in migration 2:

$ rake db:migrate
(in .... )
== 2 CreateVendors: migrating =======================================
rake aborted!
Validation failed: Rating can't be blank

(See full trace by running task with --trace)


Oops! We’re validating something that doesn’t exist yet!!

Rails saves the day again, because each validation_ declaration has an :if parameter, which should be a Symbol or Procedure to run. If this symbol or procedure method returns true, then the validation is run.

So our problem can be solved by changing the model validation to read:

validates_presence_of :rating, :on => :create, :message => "can't be blank",
:if => Proc.new { |record| record.respond_to? :rating }


responds_to asks the Rails model: “Do you have a method named ‘rating’?”. Because Rails magically creates methods for every column an entity has in the database, Rails will either have that method (because it has been defined in the database), or not (because no such column exists). So we only run the validation if we have the column, avoiding the mess with trying to validate data that doesn’t make sense (yet).

The release of a new book from one of my favorite authors gave me incentive to write this list.

Techno-geekery Fiction

Because I read so quickly, for me to spend any length of time with a book requires the book to be pretty massive… as in around a thousand pages. These books fit that bill, in a very technological setting (or diving into very technical topics)

• Neal Stephenson’s Cryptonomicon: Ton of fun if you’re interested in cryptography (from WWII and Bentley Park to modern day Public Key Encryption) and history
• Neal Stephenson’s Snow Crash: A world spun with great care: a very interesting take on the Internet (think avatars like Second Life or your favorite Massively Online Multiplayer Game, and a very interesting world outside the Internet – with mobsters, swords, a trek north on a tricked out motorbike. And skate-punks too.
• Neal Stephenson’s Anathem: I can’t wait to check this book out (September 9, 2008)!
• Neal Stephenson’s Baroque Cycle: composed of 3 1,000 page books, this took me a whole summer. A romp through Medieval Europe, with side examinations of trade, money… all kinds of good stuff. Oh right… and it has some of the history of the families from Cryptonomicon. Baroque Cycle Book 1 (Quicksilver), Baroque Cycle Book 2 (Confusion), Baroque Cycle Book 3 (System Of The World)

Geek “History”

• The Cuckoo’s Egg: A hippy-ish astronomer turned sys admin/hacker tracks hackers through all kinds of early 1990s systems to… you guessed it… East Germany. All real.
• Microserfs: Were you paying attention to the programming technologies coming out of Apple and Microsoft in say 1992-1993? (Ie: Do you remember yack about Pink or OpenDoc?) If so this book will bring back a lot of memories… and is just a good story about work, family, and companionship.

Geek Philosophy

• Things A Computer Scientist Rarely Talks About: A lecture and question and answer format where one of computer science’s great writers talks about his Christianity, his experiences writing a book where he analyzes verse 3:16 of every book in the Bible, etc. Fascinating.

Enjoy!

I’m starting something new on this blog – along with my normal content, I’ll give out book recommendations on topics or languages. I’ve got a pretty big bookshelf, and i pick the best of the best and list them here.

So, now on with my first topic: Python!

• Python In A NutshellA BIG nutshell, a great reference book. The sections on performance profiling and testing are particularly excellent. When one of my alumni went to Hong Kong to do Python work, I gave him my first edition copy of this book.
• Dive Into PythonThis is online for free at Dive Into Python.org. The book is meant to help experienced programmers ramp up into Python by taking examples and ripping them apart to see how they work
• Python Cookbook15 minutes after cracking this book open I learned so much about what Python can do, Python idioms, etc. Excellent if you’re coming from another language too and just need to learn the lay of the land

Hope this helps speed people on their Python adventures!