WebDriver and Select Boxes
This one had me puzzled for a while as I never took the time to sit down and read the documentation fully... I decided to look at this again after seeing the issue appear on the WebDriver mailing list. How to you use select and option html elements? Below is Python demo.
$ python
Python 2.6.2 (release26-maint, Apr 19 2009, 01:56:41)
[GCC 4.3.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from webdriver_firefox.webdriver import FirefoxLauncher
>>> from webdriver_firefox.webdriver import WebDriver
>>> d = WebDriver()
>>> d.get("http://cassandra.appspot.com/")
>>> e = d.find_elements_by_xpath(
"/html/body/div[@id='container']/div[@id='search']/form[@id='searchForm']/div/select")
>>> r = e[0]
>>> t = r.find_elements_by_tag_name("option")
>>> t
[<webdriver_firefox.webelement.WebElement object at 0x8dd778c>,
<webdriver_firefox.webelement.WebElement object at 0x8dd772c>,
<webdriver_firefox.webelement.WebElement object at 0x8dd77ac>,
<webdriver_firefox.webelement.WebElement object at 0x8dd77ec>]
>>> for i in t:
... print i.get_text()
...
Artist
Location
last.fm Username
Venue
>>> t[2].set_selected()
>>>
Now in your WebDriver browser session the option box has changed to "last.fm Username". Excuse the variable names but I wanted to make a note before I lost the code.
Odd Google App Engine Issue
I was having issues getting a url with urlfetch.fetch(url), it kept failing with:
[snip]
File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 241, in fetch
return rpc.get_result(allow_truncated)
File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 388, in get_result
self.check_success(allow_truncated)
File "/home/channam/Code/python/google_appengine/google/appengine/api/urlfetch.py", line 356, in check_success
raise DownloadError(str(e))
DownloadError: ApplicationError: 2
A little bit of poking found that the issue was caused by having a space in the url, something which I'm fairly certain was ok on early versions of GAE. Oh well you live and learn.
bit.ly for the win
I got my Google App Engine library featured on the list of entries for bit.ly's competition see bit.ly competition. Admittedly its a small bit of code but I hope someone might find a use for it.
But I`m still waiting for swag 
Forms in App Engine
A handy hint from an on the ball App Engine fella: how to extend the StringProperty class so that it will render as a password field
Twitter Bot
While bored I wrote the following python twitter bot. It gets the rss feed from uk hot deals and then tweets the deals. Its pretty basic as it just checks the top item on the list instead of doing dates and times. It uses python-twitter-0.5.
The script is run by cron currently every minute.
#!/usr/bin/env python
import twitter
import urllib
from xml.dom import minidom
import simplejson
api = twitter.Api(username='username', password='passwd')
def shorten(param):
""" Using bit.ly to shorten the url """
url = param
request = "http://api.bit.ly/shorten?version=2.0.1&longUrl="
request += url
request += "&login=username&apiKey=APIKEY"
# fire off request for bit.ly
sock = urllib.urlopen(request)
json = sock.read()
sock.close()
# get the json
json = simplejson.loads(json)
return json['results'][url]['shortUrl']
# get the rss feed
sock = urllib.urlopen("http://www.hotukdeals.com/rss/hot")
rss = sock.read()
sock.close()
# parse the rss into ready xml
xmldoc = minidom.parseString(rss)
# eextract our results
items = xmldoc.getElementsByTagName('item')
result = items[0].getElementsByTagName('title')[0].childNodes[0].nodeValue
result += " " +shorten(items[0].getElementsByTagName('link')[0].childNodes[0].nodeValue)
# open the temp file and read in old value
f = open('/tmp/workfile', 'w+')
existing = f.read()
existing = existing.replace('\n', '')
unicode_result = unicode(result)
# if the new value from the top of the rss feed is different tweet and record it
if unicode_result.encode('utf-8') != existing:
unicode_string = unicode(result)
f.write(unicode_string.encode('utf-8'))
if len(result) > 0:
api.PostUpdate(result)
f.close()
Dbus and Banshee
A little old news but fun. Add a track to your play queue using python:
import dbus
bus = dbus.SessionBus()
player_queue = bus.get_object("org.bansheeproject.Banshee",
"/org/bansheeproject/Banshee/SourceManager/PlayQueue")
player_queue.EnqueueUri("/home/channam/Music/Jonathon Coulton/Code Monkey.mp3",True)
App Engine and utf-8 Encoding
You may or may not have seen the error:
<type 'exceptions.UnicodeDecodeError'>: 'ascii' codec can't decode byte 0xc3 in position 2223: ordinal not in range(128)
args = ('ascii', '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Tra... Engine" />\n\t\t</div>\n\n\t</div>\n\n</body>\n\n</html>\n\n', 2223, 2224, 'ordinal not in range(128)')
encoding = 'ascii'
end = 2224
message = ''
object = '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Tra... Engine" />\n\t\t</div>\n\n\t</div>\n\n</body>\n\n</html>\n\n'
reason = 'ordinal not in range(128)'
start = 2223
This had me foxed as I fetching band names which sometimes had a fancy character in them: Motörhead for example.
To allow the string to be rendered using the following:
unicode_string = unicode(string_with_char_init)
self.response.out.write(unicode_string.encode('utf-8'))
Thats it! For App Engine that works both to render to the page or to use in urlfetch.fetch.
Woe is pylast.py
Well it tested OK my local machine but deploying it to Google created some issues. In short its just CPU hungry:
01-04 03:39PM 50.602
This request used a high amount of CPU, and was roughly 2.1 times over the average request CPU limit. High CPU requests have a small quota, and if you exceed this quota, your app will be temporarily disabled.
I removed some of the xml processing as by default it gets everything you might need. This sped it up slightly but still the fatal 500 error appeared.
Well for once it appears I was right to reinvent the wheel.
Making pylast.py play with Google App Engine
I have been playing with Google App Engine and last.fm's api for a while now. I made the standard mistake of not checking if anyone else had written a library in Python to do the hard work for me. So, after a little googling I found pyLast which is a great piece of work by Amr Hassan. After a little playing I found that it didn't play well with App Engine. This was down to the it not using urlfetch, which is no big surprise as thats a feature unique to App Engine. I also noticed it was missing the ability to fetch the date and start time of an event.
So below is a patch to App Engine up the code and fetch the date/time of an event. There is a slight oddity I have yet to figure out, the time gets appended to the date. I cant see any sane reason why currently.
Be warned this breaks the module for standard Python use unless you are have google.appengine.api kicking around in your module path.
If you wish to try out my App Engine app its over at Cassandra. Just enter the name of the artist to find out where they are playing displayed on Google Maps. Its very much an ongoing project...
diff pylast.py pylast.py.orig
37d36
< from google.appengine.api import urlfetch
286,287c285,292
< request = 'http://' + API_SERVER + API_SUBDIR + '?method=' + '&'.join(data)
< response = urlfetch.fetch(request)
---
> conn = httplib.HTTPConnection(API_SERVER)
> headers = {
> "Content-type": "application/x-www-form-urlencoded",
> 'Accept-Charset': 'utf-8',
> 'User-Agent': __name__ + '/' + __version__
> }
> conn.request('POST', API_SUBDIR, '&'.join(data), headers)
> response = conn.getresponse()
292c297
< doc = minidom.parseString(response.content)
---
> doc = minidom.parse(response)
404a410
>
1391,1392d1396
< data['date'] = self._extract(doc, 'startDate')
< data['time'] = self._extract(doc, 'startTime')
1482,1497c1486
<
< def getStartDate(self):
< """Returns the start date of the event """
<
< return self._getCachedInfo('date')
<
< def getStartTime(self):
< """Returns the start time of the event """
<
< return self._getCachedInfo('time')
<
< def getReviewCount(self):
< """Returns the number of available reviews for this event. """
<
< return self._getCachedInfo('reviews')
<
---
>