Webhook POST data


#1

Just saw the webhook function, if there is a way to program it to send the last record’s date_scraped, it will be handy to monitor any ‘outdated’ scraper


#2

I’d be up for adding data to the webhook payload. Unfortunately you wouldn’t be able to do what you describe though.

date_scraped is a concept only known to your scraper, not to morph.io. Furthermore morph.io doesn’t know what the “last record” is because the database is just a database, it doesn’t have last modified metadata on rows.


#3

Can we do select max(date_scraped) from data, then output as data and send via webhook?


#4

may work with JSONP callback


#5

Not all scrapers have date_scraped as a database field. What would we do for them?