Debugging Urllib2 in Python

I was having some trouble debugging an HTTP request using urllib2 in Python which did some unorthodox things with HTTP headers. The urllib2 module itself doesn't have much debug facility. You can see response headers by looking at the hdrs attribute on the exception like this

try:
  opener.open(url)
except urllib2.HTTPError, e:
  print e.hdrs

But in order to see the actual request string, you've got to go a layer deeper. Urllib2 runs on top of Httplib, so do this before you make your request

import httplib
httplib.HTTPConnection.debuglevel = 1