Using python 3 with morph scraperwiki fork

I’m having problems getting python 3 scrapers to save to the database on

I set my scraper runtime to python-3.4.3, then install the fork of the scraperwiki library with this:

-e git+

…but when I run, this fork complains about the missing urllib2 (which no longer exists in python 3.)

The latest version of the scraperwiki library on pypi is python 3 compatible, but doesn’t seem to be compatible (it saves to scraperwiki.sqlite, whereas I think requires it to save to data.sqlite.)

Any help greatly appreciated. I’d much prefer to use python 3.

1 Like

You’re right - all the fork does is change the database filename.

The excellent @otherchirps has a scraper that gets around this:

Let us know if that works for you too.

Great, yes – that’s fixed it. Thanks, @henare and @otherchirps

1 Like

I ran into this again, so I’ve opened a PR.