My Scrapers are all stuck

1

Hi,
My scrapers are all stuck and i cant stop them. Can You please unstuck them?
Thank you

2

Sorry for the problem @arasabbasi! They’re all fixed now.

3

Hi henare,

thank you for your help. You dont need to be sorry, I am grateful, that morph.io exists. It seems like, that scrapers 1-3 got stuck again :confused:. I programmed scrapers 4-6 with 1 h maximum runtime to avoid that unlimited stuck. I apologize in forward, but can you please stop scrapers 1-3 again please :smile:? I will update their code so that they have a 1 h runtime.

Thank you very much.

4

@arasabbasi they’re moving again :thumbsup:

5

Hi henare,

I really dont know, what i do wrong :frowning: .

now the scrapers with 1h limit are stuck too?!

How can avoid this?

Sry

6

You’re not doing anything wrong. I don’t know why it keeps happening with your scrapers, sorry.

I’ve kicked the queue in the pants again.

7

It’s been happening again because we’ve got 2 scrapers trying to log heaps of output again. One’s up to half a million lines and the other is over three quarters of a million lines :disappointed:

8

Wow! Ok :slight_smile:

.Well thats life I guess. Tbh, i let them scrape now for 6 h. 5/6 are stuck again. I am thinking of removing the 6 h limit, because it seems that the scrapers work still after they are “stuck”. Maybe the plattform just loses the Handler to the process of the “stuck” scraper, so the log is not read but the process is still running. If the scraper is aborted anyway after 24 h, then it would make sense to let the scrapers run “blind”.

What do you think?

9

Hi henare,

it seems that the scrapers in fact run even you cant stop them anymore or when the log is not refreshed.

So i increased the 6 h limit to 23 h.

10

Henare,

can you please stop my scrapers please :)?

11

Done :disappointed: One day I’ll get around to investigating my your scrapers always have problems (I’m supposed to be on holiday now).