I’m trying to automate tasks for a client but most of the site can detect my script as the bot, I will be more than glad if anyone helps me make a better tool to make puppeteer undetectable. I have tried puppeteer-extra and stealth-plugin somehow some website still able to figure out that script is being used.
Recently, I wanted to scrape plane ticket prices from a few webpages to find the cheapest flight available. Unfortunately, these websites had very good web scraping protection and I kept getting blocked.
This led me to try and build an undetectable web scraper. In this article, I’ll show you how I did it.
Below topic showed how you can scrape data from a webpage whilst being very hard to detect.
- Please note that scraping data from a web page is an ethical grey area and you should not use it on protected or copyrighted data.
Full tutorial:
https://medium.com/better-programming/scrape-the-web-with-puppeteer-in-node-js-577168831612
!