Yak Shaving

So in my pursuit of the RoR I thought I would share the process for debugger using Ruby 1.9.*

Running/Installing the Debugger

Start the server:

$ rails server --debugger

The following might appear:

$ rails server --debugger

=> Booting WEBrick
=> Rails 3.0.5 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
You need to install ruby-debug to run the server in debugging mode. With gems, use 'gem install ruby-debug'
Exiting

If so install the debug gem thus:

sudo gem install ruby-debug19

The 19 is important as without it you will receive the following error:

$ sudo gem install ruby-debug
Building native extensions.  This could take a while...
ERROR:  Error installing ruby-debug:
	ERROR: Failed to build gem native extension.

        /usr/bin/ruby1.9.1 extconf.rb
Can't handle 1.9.x yet
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers.  Check the mkmf.log file for more
details.  You may need configuration options.

Provided configuration options:
	--with-opt-dir
	--without-opt-dir
	--with-opt-include
	--without-opt-include=${opt-dir}/include
	--with-opt-lib
	--without-opt-lib=${opt-dir}/lib
	--with-make-prog
	--without-make-prog
	--srcdir=.
	--curdir
	--ruby=/usr/bin/ruby1.9.1

Gem files will remain installed in /usr/lib/ruby/gems/1.9.1/gems/linecache-0.43 for inspection.
Results logged to /usr/lib/ruby/gems/1.9.1/gems/linecache-0.43/ext/gem_make.out

Now enable the 1.9 gem, by uncommenting line 18 (at least in my file):

gem 'ruby-debug19', :require => 'ruby-debug'

To use the newly installed debugger fire up the server:

$ rails server --debugger

Debugging

To pause the executing code add debugger to pause the running server and drop you into an interactive shell.

class HomeController < ApplicationController
  def index
    @events = 'meh!'
    debugger
  end
end

Causes the following in the terminal running the server:

$ rails server --debugger
=> Booting WEBrick
=> Rails 3.0.5 application starting in development on http://0.0.0.0:3000
=> Call with -d to detach
=> Ctrl-C to shutdown server
=> Debugger enabled
[2011-03-05 22:46:53] INFO  WEBrick 1.3.1
[2011-03-05 22:46:53] INFO  ruby 1.9.2 (2010-08-18) [x86_64-linux]
[2011-03-05 22:46:53] INFO  WEBrick::HTTPServer#start: pid=4461 port=3000
/usr/lib/ruby/gems/1.9.1/gems/actionpack-3.0.5/lib/action_controller/metal/implicit_render.rb:5
default_render unless response_body
(rdb:2)

You can then call methods to inspect the current code.

This is a lovely solution and compared to having to buy an IDE to get a similar debugging functionality (yes, yes pdb, but meh) in Python this is pure win. As usual Ruby Guides have an excellent run down tutorial at http://guides.rubyonrails.org/debugging_rails_applications.html.

So thought it was time to upgrade and start Ruby on Rails on the latest stable version. First problem I hit was the latest Rails packaged for Ubuntu Maverick is 1.8 Ruby. So if you don’t mind being behind its fine.

What follows is how to get the latest greatest Rails on 1.9.2 Ruby.

sudo apt-get install ruby1.9.1-full

This gets you latest packaged Ruby, note its 1.9.1. After install you will have no “ruby” command so symlink up ruby to the actual binary thus:

$ ruby --version
The program 'ruby' is currently not installed.  You can install it by typing:
sudo apt-get install ruby
$ ruby1.9.1 --version
ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-linux]
$ sudo ln -s /usr/bin/ruby1.9.1 /usr/bin/ruby
$ ruby --version
ruby 1.9.2p0 (2010-08-18 revision 29036) [x86_64-linux]

Now we are rocking the latest Ruby using the command ruby.

Next install gems from rubygems-1.6.0.tgz. Download and run the installer:

$ wget http://production.cf.rubygems.org/rubygems/rubygems-1.6.0.tgz
$ tar xvzf rubygems-1.6.0.tgz
$ cd rubygems-1.6.0
$ sudo ruby setup.rb
$ sudo ln -s /usr/bin/gem1.9.1 /usr/bin/gem

Yay all installed. Installer the updater as per the instructions printed in the installer bit:

$ sudo gem install rubygems-update
Fetching: rubygems-update-1.6.0.gem (100%)
Successfully installed rubygems-update-1.6.0
1 gem installed
Installing ri documentation for rubygems-update-1.6.0...
Installing RDoc documentation for rubygems-update-1.6.0...
$ sudo update_rubygems
RubyGems 1.6.0 installed

Finally install rails:

$ sudo gem install rails

Before building your project chances are you need to install sqlite3 deps to head off any bundle errors.

$ sudo apt-get install libsqlite3-dev

This will stop:

checking for sqlite3.h... no
sqlite3.h is missing. Try 'port install sqlite3 +universal'

Now you are ready to roll.

I am currently refactoring a project I’m actively developing. The initial development was not test driven but more “Oooh this is cool, hack, hack hack”. I’m quite a fan of the idea of premature optimisation being a bad thing, although sometimes this can be an excuse for laziness.

So I find myself refactoring and creating tests to prove the new code works. This led me to need fixtures or exmaple data to test against. As I already had data in the database creating said fixtures was a easy task:

manage.py dumpdata app_name.model_name

Where app_name in the name of the application you create in Django and the model name is the defined database object usually in models.py. This command dumps the structure of the object out to the command line in json format. To use it create a fixtures directory in your application directory and add the json file. Or the easy way:

manage.py dumpdata app_name.model_name > fixtures/fixtures.json

The file name doesn’t matter.

While pottering about I found SimpleGeo.

I’m still struggling a little for a use case for this. It’s cool and that’s enough for me, yes I am that fickle.

Its a simple enough idea and well implemented. You sign up for free. The free account gives you a single “layer”. A layer is a collection of data points that you wish to map. Paid for accounts allow more layers but as I’m only playing I’m not about to find out. Python is well supported but not very well documented.

http://help.simplegeo.com/faqs/api-documentation/clients-and-example-code contains what you need to get the libs downloaded and running. From here on I’m going to assume you have got it installed and running.

Creating a record:

from simplegeo import Record
r = Record('layername', '1', '53.762282', '-0.355662')
r

client.add_record(r)
b = client.add_record(r)
print client.get_record('layername', '1')
{'created': 1270653503, 'geometry':
{'type': 'Point', 'coordinates': [-0.35566199999999998, 53.762281999999999]},
'properties': {'layer': 'layername', 'type': 'object'},
'layerLink': {'href': 'http://api.simplegeo.com/0.1/layer/layername.json'},
'type': 'Feature', 'id': '1', 'selfLink':
{'href': 'http://api.simplegeo.com/0.1/records/layername/1.json'}}

The code above creates and saves a record on Simple Geo’s servers and fetches the same record. Its all pretty simple stuff. Where it gets cool is using get_nearby. This allows you to send a location and get back all data points near that location.

print client.get_nearby(layer='layername',arg='53.762281,-0.3456')
{'type': 'FeatureCollection', 'features': [
{'distance': 355.3372151292516, 'created': 1270659182, 'geometry':
{'type': 'Point', 'coordinates': [-0.35099999999999998, 53.762281999999999]},
'properties': {'layer': 'layername', 'type': 'place'}, 'layerLink':
{'href': 'http://api.simplegeo.com/0.1/layer/layername.json'},
'type': 'Feature', 'id': '1', 'selfLink':
{'href': 'http://api.simplegeo.com/0.1/records/layername/1.json'}},
{'distance': 960.75183289629251, 'created': 1271073482, 'geometry':
{'type': 'Point', 'coordinates': [-0.33100000000000002, 53.762281999999999]},
'properties': {'layer': 'layername', 'type': 'place'},
'layerLink': {'href': 'http://api.simplegeo.com/0.1/layer/layername.json'},
'type': 'Feature', 'id': 'TheAdelphi', 'selfLink':
{'href': 'http://api.simplegeo.com/0.1/records/layername/TheAdelphi.json'}}]}

Pushing the coolness a bit further is get_density. This takes a location and using Sky Hooks’ Spot Rank will predict the density of people in the supplied location.

print client.get_density('51.5019', '-0.1189', 'tue')

Also worth a mention is the method get_history. This will return the last 20 locations of the record with the supplied id.

print client.get_history('layername', '1')
{'type': 'GeometryCollection', 'geometries': [
{'type': 'Point', 'coordinates': [-0.35099999999999998, 53.762281999999999],
'created': 1270659182},
{'type': 'Point', 'coordinates': [-0.35099999999999998, 53.762281999999999],
'created': 1270659072},
{'type': 'Point', 'coordinates': [-0.35566199999999998, 53.762281999999999],
'created': 1270653503}]}

While this is all very cool the only gotcha I have found is that longitude and latitude can be the wrong way round. So if you are getting no hits check the long and lat from get_record are the right way round. This is mentioned in the documentation on the Simple Geo’s site.

Another super cool feature is the dashboard offered when you login to Simple Geo’s site. There is a Google maps instance which you can use to find you data on. Handy for debugging.

Due to the almost total lack of documentation I worked out most of the above using source http://github.com/simplegeo/python-simplegeo/blob/master/simplegeo/__init__.py. Its simple enough to read.

Once I find a use for this I will write up some stuff. In the mean time if you get stuck give me a shout, chances you will be stuck at the same point I was…

This one had me puzzled for a while as I never took the time to sit down and read the documentation fully… I decided to look at this again after seeing the issue appear on the WebDriver mailing list. How to you use select and option html elements? Below is Python demo.

$ python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from webdriver_firefox.webdriver import FirefoxLauncher
>>> from webdriver_firefox.webdriver import WebDriver
>>> d = WebDriver()
>>> d.get("http://cassandra.appspot.com/")
>>> e = d.find_elements_by_xpath(
"/html/body/div[@id='container']/div[@id='search']/form[@id='searchForm']/div/select")
>>> r = e[0]
>>> t = r.find_elements_by_tag_name("option")
>>> t
[<webdriver_firefox.webelement.WebElement object at 0x8dd778c>,
<webdriver_firefox.webelement.WebElement object at 0x8dd772c>,
<webdriver_firefox.webelement.WebElement object at 0x8dd77ac>,
<webdriver_firefox.webelement.WebElement object at 0x8dd77ec>]
>>> for i in t:
...     print i.get_text()
...
Artist
Location
last.fm Username
Venue
>>> t[2].set_selected()
>>>

Now in your WebDriver browser session the option box has changed to “last.fm Username”. Excuse the variable names but I wanted to make a note before I lost the code.

I was having issues getting a url with urlfetch.fetch(url), it kept failing with:

[snip]
  File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 241, in fetch
    return rpc.get_result(allow_truncated)
  File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 388, in get_result
    self.check_success(allow_truncated)
  File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 356, in check_success
    raise DownloadError(str(e))
DownloadError: ApplicationError: 2

A little bit of poking found that the issue was caused by having a space in the url, something which I’m fairly certain was ok on early versions of GAE. Oh well you live and learn.

I got my Google App Engine library featured on the list of entries for bit.ly’s competition see bit.ly competition. Admittedly its a small bit of code but I hope someone might find a use for it.

But I`m still waiting for swag

A handy hint from an on the ball App Engine fella: how to extend the StringProperty class so that it will render as a password field

While bored I wrote the following python twitter bot. It gets the rss feed from uk hot deals and then tweets the deals. Its pretty basic as it just checks the top item on the list instead of doing dates and times. It uses python-twitter-0.5.

The script is run by cron currently every minute.

#!/usr/bin/env python

import twitter
import urllib
from xml.dom import minidom
import simplejson

api = twitter.Api(username='username', password='passwd')

def shorten(param):
	""" Using bit.ly to shorten the url """
	url = param
	request = "http://api.bit.ly/shorten?version=2.0.1&longUrl="
	request += url
	request += "&login=username&apiKey=APIKEY"

	# fire off request for bit.ly
	sock = urllib.urlopen(request)
	json = sock.read()
	sock.close()

	# get the json
	json = simplejson.loads(json)
	return json['results'][url]['shortUrl']

# get the rss feed
sock = urllib.urlopen("http://www.hotukdeals.com/rss/hot")
rss = sock.read()
sock.close()  

# parse the rss into ready xml
xmldoc = minidom.parseString(rss)

# eextract our results
items = xmldoc.getElementsByTagName('item')
result = items[0].getElementsByTagName('title')[0].childNodes[0].nodeValue
result += " " +shorten(items[0].getElementsByTagName('link')[0].childNodes[0].nodeValue)

# open the temp file and read in old value
f = open('/tmp/workfile', 'w+')
existing = f.read()
existing = existing.replace('\n', '')

unicode_result = unicode(result)

# if the new value from the top of the rss feed is different tweet and record it
if unicode_result.encode('utf-8') != existing:
	unicode_string = unicode(result)
	f.write(unicode_string.encode('utf-8'))
	if len(result) > 0:
		api.PostUpdate(result)

f.close()

A little old news but fun. Add a track to your play queue using python:

        import dbus
        bus = dbus.SessionBus()
        player_queue = bus.get_object("org.bansheeproject.Banshee",
        "/org/bansheeproject/Banshee/SourceManager/PlayQueue")
        player_queue.EnqueueUri("/home/channam/Music/Jonathon Coulton/Code Monkey.mp3",True)