App Engine and utf-8 Encoding
You may or may not have seen the error:
<type ‘exceptions.UnicodeDecodeError’>: ‘ascii’ codec can’t decode byte 0xc3 in position 2223: ordinal not in range(128)
args = (‘ascii’, ‘<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Tra… Engine” />\n\t\t</div>\n\n\t</div>\n\n</body>\n\n</html>\n\n’, 2223, 2224, ‘ordinal not in range(128)’)
encoding = ‘ascii’
end = 2224
message = ”
object = ‘<!DOCTYPE html PUBLIC “-//W3C//DTD XHTML 1.0 Tra… Engine” />\n\t\t</div>\n\n\t</div>\n\n</body>\n\n</html>\n\n’
reason = ‘ordinal not in range(128)’
start = 2223
This had me foxed as I fetching band names which sometimes had a fancy character in them: Motörhead for example.
To allow the string to be rendered using the following:
unicode_string = unicode(string_with_char_init)
self.response.out.write(unicode_string.encode('utf-8'))
Thats it! For App Engine that works both to render to the page or to use in urlfetch.fetch.
