HACKER Q&A
📣 obiefernandez

Anyone scraping in the user's browser instead of the server?


I was recently trying to help a friend working on a project that scrapes a well-known ecommerce site, and they were worried about their server getting blocked. (Probably a valid concern.) The data they're trying to get is super simple and actually helps drive sales to that site, so I don't see an ethical problem, per se. I suggested that maybe it would be possible to put the scraping logic in JS code that runs on the user's browser, so that the scraping operations, when they happen, just happen from different places in the world.

Anyone know if this is feasible? Apologies in advance if this is a stupid question, but I'm finding it impossible to Google.

EDIT: Okay maybe not so impossible to Google once I realized the challenge is primarily with CORS. Leaving this up just in case anyone has additional insight.


  👤 chovyfu Accepted Answer ✓
just use a proxy