I’m still struggling a little for a use case for this. It’s cool and that’s enough for me, yes I am that fickle.

Its a simple enough idea and well implemented. You sign up for free. The free account gives you a single “layer”. A layer is a collection of data points that you wish to map. Paid for accounts allow more layers but as I’m only playing I’m not about to find out. Python is well supported but not very well documented.

http://help.simplegeo.com/faqs/api-documentation/clients-and-example-code contains what you need to get the libs downloaded and running. From here on I’m going to assume you have got it installed and running.

Creating a record:

from simplegeo import Record
r = Record('layername', '1', '53.762282', '-0.355662')
r

client.add_record(r)
b = client.add_record(r)
print client.get_record('layername', '1')
{'created': 1270653503, 'geometry':
{'type': 'Point', 'coordinates': [-0.35566199999999998, 53.762281999999999]},
'properties': {'layer': 'layername', 'type': 'object'},
'layerLink': {'href': 'http://api.simplegeo.com/0.1/layer/layername.json'},
'type': 'Feature', 'id': '1', 'selfLink':
{'href': 'http://api.simplegeo.com/0.1/records/layername/1.json'}}

The code above creates and saves a record on Simple Geo’s servers and fetches the same record. Its all pretty simple stuff. Where it gets cool is using get_nearby. This allows you to send a location and get back all data points near that location.

print client.get_nearby(layer='layername',arg='53.762281,-0.3456')
{'type': 'FeatureCollection', 'features': [
{'distance': 355.3372151292516, 'created': 1270659182, 'geometry':
{'type': 'Point', 'coordinates': [-0.35099999999999998, 53.762281999999999]},
'properties': {'layer': 'layername', 'type': 'place'}, 'layerLink':
{'href': 'http://api.simplegeo.com/0.1/layer/layername.json'},
'type': 'Feature', 'id': '1', 'selfLink':
{'href': 'http://api.simplegeo.com/0.1/records/layername/1.json'}},
{'distance': 960.75183289629251, 'created': 1271073482, 'geometry':
{'type': 'Point', 'coordinates': [-0.33100000000000002, 53.762281999999999]},
'properties': {'layer': 'layername', 'type': 'place'},
'layerLink': {'href': 'http://api.simplegeo.com/0.1/layer/layername.json'},
'type': 'Feature', 'id': 'TheAdelphi', 'selfLink':
{'href': 'http://api.simplegeo.com/0.1/records/layername/TheAdelphi.json'}}]}

Pushing the coolness a bit further is get_density. This takes a location and using Sky Hooks’ Spot Rank will predict the density of people in the supplied location.

print client.get_density('51.5019', '-0.1189', 'tue')

Also worth a mention is the method get_history. This will return the last 20 locations of the record with the supplied id.

print client.get_history('layername', '1')
{'type': 'GeometryCollection', 'geometries': [
{'type': 'Point', 'coordinates': [-0.35099999999999998, 53.762281999999999],
'created': 1270659182},
{'type': 'Point', 'coordinates': [-0.35099999999999998, 53.762281999999999],
'created': 1270659072},
{'type': 'Point', 'coordinates': [-0.35566199999999998, 53.762281999999999],
'created': 1270653503}]}

While this is all very cool the only gotcha I have found is that longitude and latitude can be the wrong way round. So if you are getting no hits check the long and lat from get_record are the right way round. This is mentioned in the documentation on the Simple Geo’s site.

Another super cool feature is the dashboard offered when you login to Simple Geo’s site. There is a Google maps instance which you can use to find you data on. Handy for debugging.

Due to the almost total lack of documentation I worked out most of the above using source http://github.com/simplegeo/python-simplegeo/blob/master/simplegeo/__init__.py. Its simple enough to read.

Once I find a use for this I will write up some stuff. In the mean time if you get stuck give me a shout, chances you will be stuck at the same point I was…

$ python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from webdriver_firefox.webdriver import FirefoxLauncher
>>> from webdriver_firefox.webdriver import WebDriver
>>> d = WebDriver()
>>> d.get("http://cassandra.appspot.com/")
>>> e = d.find_elements_by_xpath(
"/html/body/div[@id='container']/div[@id='search']/form[@id='searchForm']/div/select")
>>> r = e[0]
>>> t = r.find_elements_by_tag_name("option")
>>> t
[<webdriver_firefox.webelement.WebElement object at 0x8dd778c>,
<webdriver_firefox.webelement.WebElement object at 0x8dd772c>,
<webdriver_firefox.webelement.WebElement object at 0x8dd77ac>,
<webdriver_firefox.webelement.WebElement object at 0x8dd77ec>]
>>> for i in t:
...     print i.get_text()
...
Artist
Location
last.fm Username
Venue
>>> t[2].set_selected()
>>>

Now in your WebDriver browser session the option box has changed to “last.fm Username”. Excuse the variable names but I wanted to make a note before I lost the code.

[snip]
  File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 241, in fetch
    return rpc.get_result(allow_truncated)
  File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 388, in get_result
    self.check_success(allow_truncated)
  File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 356, in check_success
    raise DownloadError(str(e))
DownloadError: ApplicationError: 2

A little bit of poking found that the issue was caused by having a space in the url, something which I’m fairly certain was ok on early versions of GAE. Oh well you live and learn.

But I`m still waiting for swag

The script is run by cron currently every minute.

#!/usr/bin/env python

import twitter
import urllib
from xml.dom import minidom
import simplejson

api = twitter.Api(username='username', password='passwd')

def shorten(param):
	""" Using bit.ly to shorten the url """
	url = param
	request = "http://api.bit.ly/shorten?version=2.0.1&longUrl="
	request += url
	request += "&login=username&apiKey=APIKEY"

	# fire off request for bit.ly
	sock = urllib.urlopen(request)
	json = sock.read()
	sock.close()

	# get the json
	json = simplejson.loads(json)
	return json['results'][url]['shortUrl']

# get the rss feed
sock = urllib.urlopen("http://www.hotukdeals.com/rss/hot")
rss = sock.read()
sock.close()  

# parse the rss into ready xml
xmldoc = minidom.parseString(rss)

# eextract our results
items = xmldoc.getElementsByTagName('item')
result = items[0].getElementsByTagName('title')[0].childNodes[0].nodeValue
result += " " +shorten(items[0].getElementsByTagName('link')[0].childNodes[0].nodeValue)

# open the temp file and read in old value
f = open('/tmp/workfile', 'w+')
existing = f.read()
existing = existing.replace('\n', '')

unicode_result = unicode(result)

# if the new value from the top of the rss feed is different tweet and record it
if unicode_result.encode('utf-8') != existing:
	unicode_string = unicode(result)
	f.write(unicode_string.encode('utf-8'))
	if len(result) > 0:
		api.PostUpdate(result)

f.close()

        import dbus
        bus = dbus.SessionBus()
        player_queue = bus.get_object("org.bansheeproject.Banshee",
        "/org/bansheeproject/Banshee/SourceManager/PlayQueue")
        player_queue.EnqueueUri("/home/channam/Music/Jonathon Coulton/Code Monkey.mp3",True)

<type ‘exceptions.UnicodeDecodeError’>: ‘ascii’ codec can’t decode byte 0xc3 in position 2223: ordinal not in range(128)
args = (‘ascii’, ‘<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Tra… Engine” />\n\t\t</div>\n\n\t</div>\n\n</body>\n\n</html>\n\n’, 2223, 2224, ‘ordinal not in range(128)’)
encoding = ‘ascii’
end = 2224
message = ”
object = ‘<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Tra… Engine” />\n\t\t</div>\n\n\t</div>\n\n</body>\n\n</html>\n\n’
reason = ‘ordinal not in range(128)’
start = 2223

This had me foxed as I fetching band names which sometimes had a fancy character in them: Motörhead for example.

To allow the string to be rendered using the following:

unicode_string = unicode(string_with_char_init)
self.response.out.write(unicode_string.encode('utf-8'))

Thats it! For App Engine that works both to render to the page or to use in urlfetch.fetch.

01-04 03:39PM 50.602

This request used a high amount of CPU, and was roughly 2.1 times over the average request CPU limit. High CPU requests have a small quota, and if you exceed this quota, your app will be temporarily disabled.

I removed some of the xml processing as by default it gets everything you might need. This sped it up slightly but still the fatal 500 error appeared.

Well for once it appears I was right to reinvent the wheel.

So below is a patch to App Engine up the code and fetch the date/time of an event. There is a slight oddity I have yet to figure out, the time gets appended to the date. I cant see any sane reason why currently.

Be warned this breaks the module for standard Python use unless you are have google.appengine.api kicking around in your module path.

If you wish to try out my App Engine app its over at Cassandra. Just enter the name of the artist to find out where they are playing displayed on Google Maps. Its very much an ongoing project…

diff pylast.py pylast.py.orig
37d36
< from google.appengine.api import urlfetch
286,287c285,292
< 			request = 'http://' + API_SERVER + API_SUBDIR + '?method=' + '&'.join(data)
< 			response = urlfetch.fetch(request)
---
> 			conn = httplib.HTTPConnection(API_SERVER)
> 			headers = {
> 				"Content-type": "application/x-www-form-urlencoded",
> 				'Accept-Charset': 'utf-8',
> 				'User-Agent': __name__ + '/' + __version__
> 				}
> 			conn.request('POST', API_SUBDIR, '&'.join(data), headers)
> 			response = conn.getresponse()
292c297
< 		doc = minidom.parseString(response.content)
---
> 		doc = minidom.parse(response)
404a410
>
1391,1392d1396
< 		data['date'] = self._extract(doc, 'startDate')
< 		data['time'] = self._extract(doc, 'startTime')
1482,1497c1486
<
< 	def getStartDate(self):
< 		"""Returns the start date of the event """
<
< 		return self._getCachedInfo('date')
<
< 	def getStartTime(self):
< 		"""Returns the start time of the event """
<
< 		return self._getCachedInfo('time')
<
< 	def getReviewCount(self):
< 		"""Returns the number of available reviews for this event. """
<
< 		return self._getCachedInfo('reviews')
<
---
>