My scraper stuck, what to do?

One of my scrapers is stuck:
Any chance of getting it killed?


Actually, I seem to have a few others that have been trying to run for more than a day under the wdiv-scrapers org. Can you just kill anything under that has been queued for > 1 day when you get a chance to look at this.

Hi everyone. Thanks for reporting this @chris48s and @LoveMyData.

I’ve just checked the queue and we’ve got a massive backlog. I’m investigating the cause and will try to get things moving again.

The problem is this, again:

I’ve cleaned up the stopped containers and now the queue just has to run through. Which is exacerbated by this


I’m manually checking the queue and loading jobs to try and clear things up a bit.

I’ve hit the limit of what I can do manually. We’ll just have to wait for the jobs to finish. Currently there’s about 1,000 to go. Some are not even scheduled to retry until a day from now.

Just had a check and my scraper has been running for 5 days. Sorry I didn’t notice before. Please can someone shake it loose?

We’re down to 400 retries so it looks like we’re getting through things slowly but surely. I’ve also poked and prodded to get the queue :fire:ed up again.

@lowndsy For using such a fantastic turn of phrase, I have personally checked and shaken down your scraper :smiley: It’s all fine now.

I’ve got some more scrapers that have been queued for more than a day:
Have the docker-tubes got all clogged up with goo again?

@chris48s now you’re just trying to out do @lowndsy :wink:

Yeah, yesterday The Tubes were a bit clogged with 800 runs waiting to retry. I got out my plunger and today’s it’s down to 200. I’ll keep an eye on it, thanks for letting me know.

In GitHub issues you can see I’ve got an idea of the root cause now. It’s just a matter of being able to allocate some time to fix it :confused:

@chris48s now you’re just trying to out do @lowndsy

Cue series of increasingly tortured analogies… :slight_smile:

Sorry to keep bugging you, but I’ve got a couple more that need a good kick up the backside :boot:

No worries. I just wish we had the time and resources to fix this properly. It’s a pain for everyone.

I’ve manually aborted the one of those scrapers that was still stuck after this morning’s general prodding.

Seems like my scrapers are a bit of a problem child at the moment
Why do bad things keep happening to good scrapers? :frowning:

Thanks for continuing to sort these out

My scraper is stuck as well

Everything seems to be stuck again today (or moving super-super slowly). We’ve lots of scrapers stuck at queued, and I don’t think any of our webhooks have been triggered today at all.

Yep - things have been running a lot more smoothly over the last couple of weeks but I’ve got a bunch of scrapers in the org that are blocked at the moment :frowning:

@roshin8 @tmtmtmtm @chris48s We’ve had another queue backlog over the last few days. I’m supposed to be on leave and @equivalentideas has been doing a good job keeping things moving in my absence. At time of writing it’s down to 325 runs so should be cleared out soon.

Another unstoppable beast here:

I’m sure this one might be my scrapers fault as there was (still is) lots of crazy session resetting… just realized I wasn’t quitting all of the sessions, pushing an update to fix that now

A gentle kick would be much appreciated

Looks unstuck to me now :slight_smile: We’ve been having some issues with the queue the last few days, but it seems to be working pretty well at the moment :thumbsup:

Hi. Haven’t raised it over the holidays, but I’ve got a lot of stuck scrapers under the wdiv-scrapers org:
Some of them have been queued for several weeks now. Any chance of getting them cleared?