New data not added to database after clearing it


#1

I made some changes to my scraper and wanted to clear the database to store the new data. I used the clear database option from settings and the data seems to be cleared.

But now when I run the scraper it says “Manually ran revision * and completed successfully about a minute ago. nothing changed in the database”

I can’t see any errors in the log.

What am I missing?

Thanks :slight_smile:

LE: it seems there is data in my database, but it just won’t show on me scraper page…


#2

I think you are seeing this issue: Nothing saving to database
Its probably just coincidence that you happen to have cleared your DB. You will probably find that your scraper writes a DB once an admin restarts the queue.


#3

I’ve done the restart. Have a go and let us know the results in here @aniri and thanks for posting your issue in here :thumbsup:

Thanks for being so helpful as always @chris48s :revolving_hearts:

Luke


#4

Thanks Luke, @equivalentideas , it worked! Just one more quick question, if I add more columns to the scraped data, will the db append the new table columns automatically to the data table or not?


#5

if I add more columns to the scraped data, will the db append the new table columns automatically to the data table

Yep - as long as you’re using the .save() method it will do that for you: https://github.com/openaustralia/scraperwiki-python/tree/morph_defaults#saving-data


#6

Thanks @chris48s :slight_smile: