I’m new to morph.io so forgive the newbie question, but what is the best way to clean out or truncate my data table between runs?
Welcome @loren
It’s probably a good idea to try and keep your data between runs as it could come in handy. Adding dates to records is a good way to track changes to data over time. Here’s an example from some scraper training we’ve done: https://morph.io/morph-test-scrapers/fire_danger_ratings_and_total_fire_bans_tutorial
If you really want to delete stuff from your database, and you’re using the scraperwiki library, you can use ScraperWiki.sqliteexecute
to run any SQL, including DELETE
or DROP
, against your database. e.g. to clean out your entire data table:
ScraperWiki.sqliteexecute("DROP TABLE data")