The Ultimate Web Scraping Course

10h 33m 19s
English
Paid

Course description

I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow on social media.

Read more about the course

In this course, I will show you how to scrape the following websites (and provide you with the code):

  • Instagram
  • Twitter
  • TikTok
  • LinkedIn
  • Zillow
  • Google Search
  • Google Maps
  • Amazon
  • Indeed
  • ESPN
  • Angi List
  • And more!

I will also explain how I found clients and gained over 25,000 followers on social media (and how you can do the same).

What's inside the course:

  • 9 hours of video materials
  • The legality of web scraping: what do you need to know?
  • Scrapers vs. hackers: what's the difference?
  • All the services I use (proxies, programming languages, servers, databases, packages for bypassing blocks)
  • How websites get their data (how APIs work)
  • My process for scraping ANY website
  • How to find the data source
  • How to avoid being blocked
  • All about proxies
  • Scraping by web frameworks (React, Next.js, static HTML/server-side rendering, ASP.NET, Java)
  • How to reverse engineer and bypass websites with good protection
  • Scraping websites with cursor-based pagination (e.g., comments on YouTube)
  • Scraping Chrome extensions to get free email addresses
  • How to scrape mobile apps
  • How to search for information about people (finding phone numbers and email addresses)
  • How to run Puppeteer on AWS Lambda
  • How to intercept requests with Puppeteer
  • Solving captchas with Puppeteer
  • How to find clients for web scraping
  • How to invoice clients
  • Product ideas based on web scraping

Disclaimers:

  • The code is written in JavaScript, and I use a Mac with VS Code.
  • I only briefly mention Puppeteer (there are much BETTER ways).
  • Web scraping is not magic. You need a source to scrape from (a website). For example, you cannot scrape people's bank accounts or something similar. That would fall under hacking, which is a different field.

Requirements:

  • You should have a good understanding of JavaScript or some other programming language to get the most out of this course.
  • If you cannot program at all, this course is not for you.
  • I do not teach the basics of programming or JavaScript in this course.

Watch Online

Join premium to watch
Go to premium
# Title Duration
1 001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024 05:27
2 002 Legality of web scraping 08:37
3 003 Scraping vs Hacking 04:35
4 004 Services we'll be using in the course 04:36
5 005 Services we'll be using in the course 2 02:44
6 006 How websites get their data 07:58
7 007 My process for Scraping Any Site 04:16
8 008 How to find where the data is coming from 04:18
9 009 Proxies 21:23
10 010 Scraping Next.js sites (angi, imdb) 21:17
11 011 ESPN - Scraping API's 16:50
12 012 Amazon - Straight up HTML page 10:36
13 013 Update on Overcoming CloudFlare Blocks 04:23
14 014 Indeed - Straight Up HTML 25:14
15 015 Sarpy - ASP.NET (long version) 46:08
16 016 Sarpy - ASP.NET Cliff Notes 16:15
17 017 Kitsap - Java Site 22:07
18 018 Google Maps 46:16
19 019 Linkedin 08:55
20 020 How to scrape Shopify 07:50
21 021 They can't hide anything from you.... 01:52
22 022 Reverse Engineering Realtor API 27:52
23 023 Lynchburg exposing Auth Token 04:03
24 024 Beacons.ai 19:45
25 025 Twitter 18:29
26 026 Instagram Scraping Update 07:38
27 027 How to Scrape Instagram Profiles 03:42
28 028 Instagram 32:37
29 029 Arcgis 16:44
30 030 Cursor Based Pagination 11:54
31 031 Scraping Mobile Apps 09:34
32 032 Scraping Zillow Data Tutorial 11:33
33 033 Skip Tracing 09:39
34 034 Getting Puppeteer Up and Running on AWS Lambda 15:39
35 035 facbleag 08:01
36 036 Getting Tiktok Videos with Puppeteer 05:15
37 037 Solving CAPTCHAs with Puppeteer 09:44
38 038 Sitemaps (crunchbase) 10:29
39 039 Update 12312024 Use puppeteer to get company pages 04:59
40 040 Google Tricks and Tips 03:49
41 041 Decoding Imperva Security Measures 02:49
42 042 Leveraging Google's Web Cache to get around Cloudflare and Datadome blocking 03:35
43 043 Scraping Kick (got-scraping + use your resources) 08:21
44 044 I found a way to scrape Ohio State Student Emails 07:15
45 045 Tiktok API 04:35
46 046 SC Courts, use undetected chrome driver 02:48
47 047 Exploring Supabase for Efficient Database Management 06:32
48 048 Where to host the scrapers_ Render.com 04:11
49 049 Quick note on Concurrency in Web Scraping 02:44
50 050 How to make money web scraping 08:48
51 051 How to get web scraping clients 35:52
52 052 What do you charge clients 10:04
53 053 What should you scrape for 12:42

Similar courses

JavaScript Algorithms and Data Structures Masterclass

JavaScript Algorithms and Data Structures Masterclass

Sources: udemy
This course crams months of computer science and interview prep material into 20 hours of video. The content is based directly on last semester of my in-person coding bootcamps,...
21 hours 47 minutes 46 seconds
Zero to Full Stack Hero

Zero to Full Stack Hero

Sources: papareact.com
PAPA React presents.. Zero to Full Stack Hero. It's NOT just another COURSE. It's the world's BEST COMMUNITY. From learning the Basics of Web Development to Mastering React!
101 hours 29 minutes 59 seconds
Mastering JavaScript Unit Testing

Mastering JavaScript Unit Testing

Sources: codewithmosh (Mosh Hamedani)
A comprehensive, beginner-friendly guide covering everything from the basics to advanced techniques. Tired of piecing together disconnected tutorials or dealing with rambling, c...
3 hours 51 minutes 31 seconds