The Ultimate Web Scraping Course

10h 33m 19s
English
Paid

Course description

I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow on social media.

Read more about the course

In this course, I will show you how to scrape the following websites (and provide you with the code):

  • Instagram
  • Twitter
  • TikTok
  • LinkedIn
  • Zillow
  • Google Search
  • Google Maps
  • Amazon
  • Indeed
  • ESPN
  • Angi List
  • And more!

I will also explain how I found clients and gained over 25,000 followers on social media (and how you can do the same).

What's inside the course:

  • 9 hours of video materials
  • The legality of web scraping: what do you need to know?
  • Scrapers vs. hackers: what's the difference?
  • All the services I use (proxies, programming languages, servers, databases, packages for bypassing blocks)
  • How websites get their data (how APIs work)
  • My process for scraping ANY website
  • How to find the data source
  • How to avoid being blocked
  • All about proxies
  • Scraping by web frameworks (React, Next.js, static HTML/server-side rendering, ASP.NET, Java)
  • How to reverse engineer and bypass websites with good protection
  • Scraping websites with cursor-based pagination (e.g., comments on YouTube)
  • Scraping Chrome extensions to get free email addresses
  • How to scrape mobile apps
  • How to search for information about people (finding phone numbers and email addresses)
  • How to run Puppeteer on AWS Lambda
  • How to intercept requests with Puppeteer
  • Solving captchas with Puppeteer
  • How to find clients for web scraping
  • How to invoice clients
  • Product ideas based on web scraping

Disclaimers:

  • The code is written in JavaScript, and I use a Mac with VS Code.
  • I only briefly mention Puppeteer (there are much BETTER ways).
  • Web scraping is not magic. You need a source to scrape from (a website). For example, you cannot scrape people's bank accounts or something similar. That would fall under hacking, which is a different field.

Requirements:

  • You should have a good understanding of JavaScript or some other programming language to get the most out of this course.
  • If you cannot program at all, this course is not for you.
  • I do not teach the basics of programming or JavaScript in this course.

Watch Online

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 53 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: 001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024

All Course Lessons (53)

#Lesson TitleDurationAccess
1
001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024 Demo
05:27
2
002 Legality of web scraping
08:37
3
003 Scraping vs Hacking
04:35
4
004 Services we'll be using in the course
04:36
5
005 Services we'll be using in the course 2
02:44
6
006 How websites get their data
07:58
7
007 My process for Scraping Any Site
04:16
8
008 How to find where the data is coming from
04:18
9
009 Proxies
21:23
10
010 Scraping Next.js sites (angi, imdb)
21:17
11
011 ESPN - Scraping API's
16:50
12
012 Amazon - Straight up HTML page
10:36
13
013 Update on Overcoming CloudFlare Blocks
04:23
14
014 Indeed - Straight Up HTML
25:14
15
015 Sarpy - ASP.NET (long version)
46:08
16
016 Sarpy - ASP.NET Cliff Notes
16:15
17
017 Kitsap - Java Site
22:07
18
018 Google Maps
46:16
19
019 Linkedin
08:55
20
020 How to scrape Shopify
07:50
21
021 They can't hide anything from you....
01:52
22
022 Reverse Engineering Realtor API
27:52
23
023 Lynchburg exposing Auth Token
04:03
24
024 Beacons.ai
19:45
25
025 Twitter
18:29
26
026 Instagram Scraping Update
07:38
27
027 How to Scrape Instagram Profiles
03:42
28
028 Instagram
32:37
29
029 Arcgis
16:44
30
030 Cursor Based Pagination
11:54
31
031 Scraping Mobile Apps
09:34
32
032 Scraping Zillow Data Tutorial
11:33
33
033 Skip Tracing
09:39
34
034 Getting Puppeteer Up and Running on AWS Lambda
15:39
35
035 facbleag
08:01
36
036 Getting Tiktok Videos with Puppeteer
05:15
37
037 Solving CAPTCHAs with Puppeteer
09:44
38
038 Sitemaps (crunchbase)
10:29
39
039 Update 12312024 Use puppeteer to get company pages
04:59
40
040 Google Tricks and Tips
03:49
41
041 Decoding Imperva Security Measures
02:49
42
042 Leveraging Google's Web Cache to get around Cloudflare and Datadome blocking
03:35
43
043 Scraping Kick (got-scraping + use your resources)
08:21
44
044 I found a way to scrape Ohio State Student Emails
07:15
45
045 Tiktok API
04:35
46
046 SC Courts, use undetected chrome driver
02:48
47
047 Exploring Supabase for Efficient Database Management
06:32
48
048 Where to host the scrapers_ Render.com
04:11
49
049 Quick note on Concurrency in Web Scraping
02:44
50
050 How to make money web scraping
08:48
51
051 How to get web scraping clients
35:52
52
052 What do you charge clients
10:04
53
053 What should you scrape for
12:42

Unlock unlimited learning

Get instant access to all 52 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Comments

0 comments

Want to join the conversation?

Sign in to comment

Similar courses

Youtube clone

Youtube clone

Sources: Nomad Coders
What is "clone coding"? Services that actually exist. Sweat. Sweat. Practical 100% educational method to learn while creating Clone coding is a way of learning while creating re...
15 hours 41 minutes 20 seconds
100 Days Of Code: The Complete Web Development Bootcamp 2024

100 Days Of Code: The Complete Web Development Bootcamp 2024

Sources: Academind Pro
Learn web development from A to Z in 100 days (or at your own pace) - from "basic" to "advanced", it's all included!
78 hours 51 minutes 55 seconds
Remix Bootcamp: Zero to Mastery

Remix Bootcamp: Zero to Mastery

Sources: zerotomastery.io
Join Remix Bootcamp to master full-stack web development. Learn from industry experts to build better websites and advance your career in web development
21 hours 2 minutes 22 seconds