Certificate verify failed

1

I’ve got a couple of scrapers which both started consistently failing 5 days ago with the error

urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:600)>

https://morph.io/wdiv-scrapers/DC-PollingStations-Leeds
https://morph.io/wdiv-scrapers/DC-PollingStations-Barnet

Both of them run on my local machine. Examples of the URLs I’m scraping are:

https://gis.leeds.gov.uk/arcgis/rest/services/LeedsData/MapServer/37/query?where=OBJECTID+LIKE+'%'&text=&objectIds=&time=&geometry=&geometryType=esriGeometryPoint&inSR=&spatialRel=esriSpatialRelIntersects&relationParam=&outFields=*&returnGeometry=true&maxAllowableOffset=&geometryPrecision=&outSR=4326&returnIdsOnly=false&returnCountOnly=false&orderByFields=OBJECTID&groupByFieldsForStatistics=&outStatistics=&returnZ=false&returnM=false&gdbVersion=&returnDistinctValues=false&f=pjson

https://open.barnet.gov.uk/api/3/action/package_show?id=polling-stations-in-barnet

You should also find both of those URLs will load in a web browser.

I should be able to work around it by switching them to use HTTP instead of HTTPS if you don’t have any suggestions, but the fact that they are contacting unrelated servers and both started failing on the same day suggests there may be a problem which is worth looking at.

Thanks,
Chris

2

Thank you for reporting this @chris48s - it’s really helpful. I think this issue is related to:

3

OK cheers. For the moment I’ve switched them to use HTTP instead which does work.

If you find a fix and want me to test it, give me a shout. I can switch them back and see what happens.