One of my scrapers is stuck:
https://morph.io/wdiv-scrapers/DC-PollingStations-Eden
Any chance of getting it killed?
Cheers
Chris
Actually, I seem to have a few others that have been trying to run for more than a day under the wdiv-scrapers org. Can you just kill anything under https://morph.io/wdiv-scrapers that has been queued for > 1 day when you get a chance to look at this.
Thanks
Hi everyone. Thanks for reporting this @chris48s and @LoveMyData.
Iāve just checked the queue and weāve got a massive backlog. Iām investigating the cause and will try to get things moving again.
The problem is this, again: https://github.com/openaustralia/morph/issues/1098
Iāve cleaned up the stopped containers and now the queue just has to run through. Which is exacerbated by this https://github.com/openaustralia/morph/issues/1099
Iām manually checking the queue and loading jobs to try and clear things up a bit.
Iāve hit the limit of what I can do manually. Weāll just have to wait for the jobs to finish. Currently thereās about 1,000 to go. Some are not even scheduled to retry until a day from now.
Just had a check and my scraper has been running for 5 days. Sorry I didnāt notice before. Please can someone shake it loose? https://morph.io/lowndsy/PositionAnalyser
Weāre down to 400 retries so it looks like weāre getting through things slowly but surely. Iāve also poked and prodded to get the queue 
ed up again.
@lowndsy For using such a fantastic turn of phrase, I have personally checked and shaken down your scraper 
Itās all fine now.
Iāve got some more scrapers that have been queued for more than a day: https://morph.io/wdiv-scrapers/
Have the docker-tubes got all clogged up with goo again?
@chris48s now youāre just trying to out do @lowndsy 
Yeah, yesterday The Tubes were a bit clogged with 800 runs waiting to retry. I got out my plunger and todayās itās down to 200. Iāll keep an eye on it, thanks for letting me know.
In GitHub issues you can see Iāve got an idea of the root cause now. Itās just a matter of being able to allocate some time to fix it 
Cue series of increasingly tortured analogiesā¦ 
Sorry to keep bugging you, but Iāve got a couple more that need a good kick up the backside 
https://morph.io/wdiv-scrapers/DC-PollingStations-Calderdale
https://morph.io/wdiv-scrapers/DC-PollingStations-Mid-Sussex
No worries. I just wish we had the time and resources to fix this properly. Itās a pain for everyone.
Iāve manually aborted the one of those scrapers that was still stuck after this morningās general prodding.
Seems like my scrapers are a bit of a problem child at the moment
https://morph.io/wdiv-scrapers/DC-PollingStations-Lancaster
Why do bad things keep happening to good scrapers? 
Thanks for continuing to sort these out
Everything seems to be stuck again today (or moving super-super slowly). Weāve lots of scrapers stuck at queued, and I donāt think any of our webhooks have been triggered today at all.
Yep - things have been running a lot more smoothly over the last couple of weeks but Iāve got a bunch of scrapers in the https://morph.io/wdiv-scrapers org that are blocked at the moment
@roshin8 @tmtmtmtm @chris48s Weāve had another queue backlog over the last few days. Iām supposed to be on leave and @equivalentideas has been doing a good job keeping things moving in my absence. At time of writing itās down to 325 runs so should be cleared out soon.
Another unstoppable beast here: https://morph.io/PuZZleDucK/VIC_Tenders
Iām sure this one might be my scrapers fault as there was (still is) lots of crazy session resettingā¦ just realized I wasnāt quitting all of the sessions, pushing an update to fix that now
A gentle kick would be much appreciated
Looks unstuck to me now 
Weāve been having some issues with the queue the last few days, but it seems to be working pretty well at the moment 
Hi. Havenāt raised it over the holidays, but Iāve got a lot of stuck scrapers under the wdiv-scrapers org: https://morph.io/wdiv-scrapers
Some of them have been queued for several weeks now. Any chance of getting them cleared?
Thanks