I was wondering if you have any plans to include R as one of the languages.
Firstly thanks so much for moving this discussion over from github.
I’m completely open to adding R as one of the supported language. It wasn’t something that we were initially planning but if it’s something that you and others would find invaluable then let’s add it!
I’ve only used R very occasionally. Can you actually do web scraping with it?
We recently had a contribution to add perl support:
This should give you an idea of what’s involved in updating the morph app itself. It really isn’t terribly much. The biggest thing is creating a docker image for the R support. Again for the perl version see here for what it looks like:
Dockerfile is basically a list of commands that need to get run to install R.
It would be fantastic if you could consider adding this support. We would merge a PR from you as soon as possible.
If you’re not in a position to do it, it’s something that we could do at some stage but I can’t promise how soon.
All the best,
Thank you for the feedback @mlandauer. Indeed, I would say that scrapping isn’t R’s forte. But it certainly does it. I would say that the biggest advantage is to make calculations at the scraper level, before storing data in a database. That would be neat if done through morph.io.
Let em take a look at the procedure you outlined above. If I don’t stall in the technical details, I can certainly give it a try and send a pull request.
Awesome. If you run into any difficulties please don’t hesitate to ask.
You can access R from Python apparently through a few different libraries, such as rpy, so maybe that’s an option?
This is obviously a very old thread but as the core of morph.io has changed quite a bit since last year it’s worth updating things for completeness.
Now, all scrapers are run using something called buildpacks which is the technology developed by Heroku to support multiple languages on their platform.
We borrow the same technology to run scrapers and any language that can be used in buildpacks can potentially be used in morph.io.
Maybe someone has written a buildpack for R already if this is something that people are still interested in having?