The Ultimate Web Scraping Course
10h 39m 26s
English
Paid
August 30, 2024
I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow on social media.
More
In this course, I will show you how to scrape the following websites (and provide you with the code):
- TikTok
- Zillow
- Google Search
- Google Maps
- Amazon
- Indeed
- ESPN
- Angi List
- And more!
I will also explain how I found clients and gained over 25,000 followers on social media (and how you can do the same).
What's inside the course:
- 9 hours of video materials
- The legality of web scraping: what do you need to know?
- Scrapers vs. hackers: what's the difference?
- All the services I use (proxies, programming languages, servers, databases, packages for bypassing blocks)
- How websites get their data (how APIs work)
- My process for scraping ANY website
- How to find the data source
- How to avoid being blocked
- All about proxies
- Scraping by web frameworks (React, Next.js, static HTML/server-side rendering, ASP.NET, Java)
- How to reverse engineer and bypass websites with good protection
- Scraping websites with cursor-based pagination (e.g., comments on YouTube)
- Scraping Chrome extensions to get free email addresses
- How to scrape mobile apps
- How to search for information about people (finding phone numbers and email addresses)
- How to run Puppeteer on AWS Lambda
- How to intercept requests with Puppeteer
- Solving captchas with Puppeteer
- How to find clients for web scraping
- How to invoice clients
- Product ideas based on web scraping
Disclaimers:
- The code is written in JavaScript, and I use a Mac with VS Code.
- I only briefly mention Puppeteer (there are much BETTER ways).
- Web scraping is not magic. You need a source to scrape from (a website). For example, you cannot scrape people's bank accounts or something similar. That would fall under hacking, which is a different field.
Requirements:
- You should have a good understanding of JavaScript or some other programming language to get the most out of this course.
- If you cannot program at all, this course is not for you.
- I do not teach the basics of programming or JavaScript in this course.
Watch Online The Ultimate Web Scraping Course
Join premium to watch
Go to premium
# | Title | Duration |
---|---|---|
1 | Is Web Scraping Legal? | 08:37 |
2 | Hackers vs Scrapers | 04:35 |
3 | My go to packages, editor, language, proxies, etc | 04:36 |
4 | How websites get their data/how API’s work | 07:58 |
5 | My Process for Scraping Any Site | 04:16 |
6 | How to find where data is coming from | 04:18 |
7 | How to set up proxies | 21:23 |
8 | How to make money web scraping | 08:48 |
9 | Scraping Next.js sites (angi, imdb) | 21:17 |
10 | Zillow - Scraping Hidden API's | 32:31 |
11 | ESPN - Scraping API's | 16:50 |
12 | Amazon - Straight up HTML page | 10:36 |
13 | Indeed - Straight Up HTML | 25:14 |
14 | Sarpy - ASP.NET (long version) | 46:08 |
15 | Sarpy - ASP.NET Cliff Notes | 16:15 |
16 | Kitsap - Java Site | 22:07 |
17 | Google Search | 13:37 |
18 | Google Maps | 46:16 |
19 | 08:55 | |
20 | Shopify | 07:50 |
21 | They can't hide anything from you..... | 01:52 |
22 | Reverse Engineering Realtor API | 27:52 |
23 | Lynchburg exposing Auth Token | 04:03 |
24 | Beacons.ai | 19:45 |
25 | 18:29 | |
26 | 32:37 | |
27 | Arcgis | 16:44 |
28 | Cursor Based Pagination (YouTube Comments) | 11:54 |
29 | How to scrape Chrome Extensions (free emails) | 08:22 |
30 | Reverse Engineering a Private API with mitmproxy | 08:25 |
31 | Scraping Mobile Apps | 09:34 |
32 | Skip Tracing | 09:39 |
33 | Getting Puppeteer Up and Running on AWS Lambda | 15:39 |
34 | Getting Tiktok Videos with Puppeteer | 05:15 |
35 | Solving CAPTCHAs with Puppeteer | 09:44 |
36 | Sitemaps (crunchbase) | 10:29 |
37 | Google Tricks | 03:49 |
38 | Decoding Imperva Security Measures | 02:49 |
39 | Leveraging Google's Web Cache to get around Cloudflare and Datadome | 03:35 |
40 | Tiktok API | 04:35 |
41 | SC Courts, use undetected chrome driver | 02:48 |
42 | Exploring Supabase for Efficient Database Management | 06:32 |
43 | Where to host the scrapers? Render.com | 04:11 |
44 | Quick note on Concurrency in Web Scraping | 02:44 |
45 | How to get web scraping clients | 35:52 |
46 | What do you charge clients?? | 10:04 |
47 | What should you scrape for $$? | 12:42 |
48 | I found a way to scrape Ohio State Student Emails | 07:15 |
Similar courses to The Ultimate Web Scraping Course
The Approachable Guide to Accessible Componentsfullstack.io
Duration 2 hours 19 minutes 45 seconds
Course
Modern JavaScriptui.dev (ex. Tyler McGinnis)
Duration 3 hours 4 minutes 3 seconds
Course
Make a Google search engine clone: JavaScript PHP and MySQLudemy
Duration 5 hours 47 minutes 11 seconds
Course
Learn JavaScript Closures (PRO)codesmith.io
Duration 5 hours 15 minutes 31 seconds
Course
70+ JavaScript Challenges: Data Structures & AlgorithmsBrad Traversy
Duration 12 hours 29 minutes 29 seconds
Course
Build Fullstack Trello clone: WebSocket, Socket IOudemy
Duration 8 hours 49 minutes 48 seconds
Course