The Ultimate Web Scraping Course
10h 33m 19s
English
Paid
Course description
I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow on social media.
Read more about the course
In this course, I will show you how to scrape the following websites (and provide you with the code):
- TikTok
- Zillow
- Google Search
- Google Maps
- Amazon
- Indeed
- ESPN
- Angi List
- And more!
I will also explain how I found clients and gained over 25,000 followers on social media (and how you can do the same).
What's inside the course:
- 9 hours of video materials
- The legality of web scraping: what do you need to know?
- Scrapers vs. hackers: what's the difference?
- All the services I use (proxies, programming languages, servers, databases, packages for bypassing blocks)
- How websites get their data (how APIs work)
- My process for scraping ANY website
- How to find the data source
- How to avoid being blocked
- All about proxies
- Scraping by web frameworks (React, Next.js, static HTML/server-side rendering, ASP.NET, Java)
- How to reverse engineer and bypass websites with good protection
- Scraping websites with cursor-based pagination (e.g., comments on YouTube)
- Scraping Chrome extensions to get free email addresses
- How to scrape mobile apps
- How to search for information about people (finding phone numbers and email addresses)
- How to run Puppeteer on AWS Lambda
- How to intercept requests with Puppeteer
- Solving captchas with Puppeteer
- How to find clients for web scraping
- How to invoice clients
- Product ideas based on web scraping
Disclaimers:
- The code is written in JavaScript, and I use a Mac with VS Code.
- I only briefly mention Puppeteer (there are much BETTER ways).
- Web scraping is not magic. You need a source to scrape from (a website). For example, you cannot scrape people's bank accounts or something similar. That would fall under hacking, which is a different field.
Requirements:
- You should have a good understanding of JavaScript or some other programming language to get the most out of this course.
- If you cannot program at all, this course is not for you.
- I do not teach the basics of programming or JavaScript in this course.
Watch Online
Watch Online The Ultimate Web Scraping Course
0:00
/ #1: 001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024
All Course Lessons (53)
| # | Lesson Title | Duration | Access |
|---|---|---|---|
| 1 | 001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024 Demo | 05:27 | |
| 2 | 002 Legality of web scraping | 08:37 | |
| 3 | 003 Scraping vs Hacking | 04:35 | |
| 4 | 004 Services we'll be using in the course | 04:36 | |
| 5 | 005 Services we'll be using in the course 2 | 02:44 | |
| 6 | 006 How websites get their data | 07:58 | |
| 7 | 007 My process for Scraping Any Site | 04:16 | |
| 8 | 008 How to find where the data is coming from | 04:18 | |
| 9 | 009 Proxies | 21:23 | |
| 10 | 010 Scraping Next.js sites (angi, imdb) | 21:17 | |
| 11 | 011 ESPN - Scraping API's | 16:50 | |
| 12 | 012 Amazon - Straight up HTML page | 10:36 | |
| 13 | 013 Update on Overcoming CloudFlare Blocks | 04:23 | |
| 14 | 014 Indeed - Straight Up HTML | 25:14 | |
| 15 | 015 Sarpy - ASP.NET (long version) | 46:08 | |
| 16 | 016 Sarpy - ASP.NET Cliff Notes | 16:15 | |
| 17 | 017 Kitsap - Java Site | 22:07 | |
| 18 | 018 Google Maps | 46:16 | |
| 19 | 019 Linkedin | 08:55 | |
| 20 | 020 How to scrape Shopify | 07:50 | |
| 21 | 021 They can't hide anything from you.... | 01:52 | |
| 22 | 022 Reverse Engineering Realtor API | 27:52 | |
| 23 | 023 Lynchburg exposing Auth Token | 04:03 | |
| 24 | 024 Beacons.ai | 19:45 | |
| 25 | 025 Twitter | 18:29 | |
| 26 | 026 Instagram Scraping Update | 07:38 | |
| 27 | 027 How to Scrape Instagram Profiles | 03:42 | |
| 28 | 028 Instagram | 32:37 | |
| 29 | 029 Arcgis | 16:44 | |
| 30 | 030 Cursor Based Pagination | 11:54 | |
| 31 | 031 Scraping Mobile Apps | 09:34 | |
| 32 | 032 Scraping Zillow Data Tutorial | 11:33 | |
| 33 | 033 Skip Tracing | 09:39 | |
| 34 | 034 Getting Puppeteer Up and Running on AWS Lambda | 15:39 | |
| 35 | 035 facbleag | 08:01 | |
| 36 | 036 Getting Tiktok Videos with Puppeteer | 05:15 | |
| 37 | 037 Solving CAPTCHAs with Puppeteer | 09:44 | |
| 38 | 038 Sitemaps (crunchbase) | 10:29 | |
| 39 | 039 Update 12312024 Use puppeteer to get company pages | 04:59 | |
| 40 | 040 Google Tricks and Tips | 03:49 | |
| 41 | 041 Decoding Imperva Security Measures | 02:49 | |
| 42 | 042 Leveraging Google's Web Cache to get around Cloudflare and Datadome blocking | 03:35 | |
| 43 | 043 Scraping Kick (got-scraping + use your resources) | 08:21 | |
| 44 | 044 I found a way to scrape Ohio State Student Emails | 07:15 | |
| 45 | 045 Tiktok API | 04:35 | |
| 46 | 046 SC Courts, use undetected chrome driver | 02:48 | |
| 47 | 047 Exploring Supabase for Efficient Database Management | 06:32 | |
| 48 | 048 Where to host the scrapers_ Render.com | 04:11 | |
| 49 | 049 Quick note on Concurrency in Web Scraping | 02:44 | |
| 50 | 050 How to make money web scraping | 08:48 | |
| 51 | 051 How to get web scraping clients | 35:52 | |
| 52 | 052 What do you charge clients | 10:04 | |
| 53 | 053 What should you scrape for | 12:42 |
Unlock unlimited learning
Get instant access to all 52 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.
Learn more about subscriptionComments
0 commentsWant to join the conversation?
Sign in to commentSimilar courses
The Creative Javascript Course
Sources: developedbyed.com
Javascript is the most popular programming language in the world! This course is ideal for anyone who wants to get started in front end development. Learn javas
17 hours 46 minutes 20 seconds
JavaScript Algorithms and Data Structures Masterclass
Sources: udemy
This course crams months of computer science and interview prep material into 20 hours of video. The content is based directly on last semester of my in-person coding bootcamps,...
21 hours 47 minutes 46 seconds
Full-Stack Web Developer Bootcamp with Real Projects
Sources: udemy
This Course covers full process of web development from scratch till deployment with domain name. We will use Node, Express, MongoDB to build Server side which
22 hours 11 minutes 13 seconds
Complete Web Developer in 2025: Zero to Mastery
Sources: udemy, zerotomastery.io
Learn to code. Get hired. This is one of the most popular, highly rated coding bootcamps online. It's also the most moderen and up-to-date. Guaranteed. You'll g
37 hours 3 minutes 37 seconds