CF

The Ultimate Web Scraping Course

10h 33m 19s
English
Paid

The Ultimate Web Scraping Course is a 53-lesson 10 hours 33 minutes self-paced course by Adrian Horning (The Web Scraping Guy). I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow…

Course facts

Lessons
53
Duration
10 hours 33 minutes
Level
All levels
Language
English
Updated
Instructor
Adrian Horning (The Web Scraping Guy)
Price
Premium

I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow on social media.

In this course, I will show you how to scrape the following websites (and provide you with the code):

  • Instagram
  • Twitter
  • TikTok
  • LinkedIn
  • Zillow
  • Google Search
  • Google Maps
  • Amazon
  • Indeed
  • ESPN
  • Angi List
  • And more!

I will also explain how I found clients and gained over 25,000 followers on social media (and how you can do the same).

What's inside the course:

  • 9 hours of video materials
  • The legality of web scraping: what do you need to know?
  • Scrapers vs. hackers: what's the difference?
  • All the services I use (proxies, programming languages, servers, databases, packages for bypassing blocks)
  • How websites get their data (how APIs work)
  • My process for scraping ANY website
  • How to find the data source
  • How to avoid being blocked
  • All about proxies
  • Scraping by web frameworks (React, Next.js, static HTML/server-side rendering, ASP.NET, Java)
  • How to reverse engineer and bypass websites with good protection
  • Scraping websites with cursor-based pagination (e.g., comments on YouTube)
  • Scraping Chrome extensions to get free email addresses
  • How to scrape mobile apps
  • How to search for information about people (finding phone numbers and email addresses)
  • How to run Puppeteer on AWS Lambda
  • How to intercept requests with Puppeteer
  • Solving captchas with Puppeteer
  • How to find clients for web scraping
  • How to invoice clients
  • Product ideas based on web scraping

Disclaimers:

  • The code is written in JavaScript, and I use a Mac with VS Code.
  • I only briefly mention Puppeteer (there are much BETTER ways).
  • Web scraping is not magic. You need a source to scrape from (a website). For example, you cannot scrape people's bank accounts or something similar. That would fall under hacking, which is a different field.

Requirements:

  • You should have a good understanding of JavaScript or some other programming language to get the most out of this course.
  • If you cannot program at all, this course is not for you.
  • I do not teach the basics of programming or JavaScript in this course.

Who teaches The Ultimate Web Scraping Course? Adrian Horning (The Web Scraping Guy)

Adrian Horning (The Web Scraping Guy) thumbnail

Adrian Horning publishes under the brand The Web Scraping Guy — focused entirely on the practical side of pulling structured data out of websites that weren't designed to give it up. His material covers the JavaScript and Node.js tooling that dominates modern scraping work (Playwright, Puppeteer, Cheerio, headless browsers, proxy rotation).

The CourseFlix listing carries The Ultimate Web Scraping Course, a paid resource aimed at developers building scraping pipelines — whether for data journalism, market research, or product features that need to consume third-party sites at scale.

What lessons are included in The Ultimate Web Scraping Course?

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 53 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: 001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024
All Course Lessons (53)
#Lesson TitleDurationAccess
1
001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024 Demo
05:27
2
002 Legality of web scraping
08:37
3
003 Scraping vs Hacking
04:35
4
004 Services we'll be using in the course
04:36
5
005 Services we'll be using in the course 2
02:44
6
006 How websites get their data
07:58
7
007 My process for Scraping Any Site
04:16
8
008 How to find where the data is coming from
04:18
9
009 Proxies
21:23
10
010 Scraping Next.js sites (angi, imdb)
21:17
11
011 ESPN - Scraping API's
16:50
12
012 Amazon - Straight up HTML page
10:36
13
013 Update on Overcoming CloudFlare Blocks
04:23
14
014 Indeed - Straight Up HTML
25:14
15
015 Sarpy - ASP.NET (long version)
46:08
16
016 Sarpy - ASP.NET Cliff Notes
16:15
17
017 Kitsap - Java Site
22:07
18
018 Google Maps
46:16
19
019 Linkedin
08:55
20
020 How to scrape Shopify
07:50
21
021 They can't hide anything from you....
01:52
22
022 Reverse Engineering Realtor API
27:52
23
023 Lynchburg exposing Auth Token
04:03
24
024 Beacons.ai
19:45
25
025 Twitter
18:29
26
026 Instagram Scraping Update
07:38
27
027 How to Scrape Instagram Profiles
03:42
28
028 Instagram
32:37
29
029 Arcgis
16:44
30
030 Cursor Based Pagination
11:54
31
031 Scraping Mobile Apps
09:34
32
032 Scraping Zillow Data Tutorial
11:33
33
033 Skip Tracing
09:39
34
034 Getting Puppeteer Up and Running on AWS Lambda
15:39
35
035 facbleag
08:01
36
036 Getting Tiktok Videos with Puppeteer
05:15
37
037 Solving CAPTCHAs with Puppeteer
09:44
38
038 Sitemaps (crunchbase)
10:29
39
039 Update 12312024 Use puppeteer to get company pages
04:59
40
040 Google Tricks and Tips
03:49
41
041 Decoding Imperva Security Measures
02:49
42
042 Leveraging Google's Web Cache to get around Cloudflare and Datadome blocking
03:35
43
043 Scraping Kick (got-scraping + use your resources)
08:21
44
044 I found a way to scrape Ohio State Student Emails
07:15
45
045 Tiktok API
04:35
46
046 SC Courts, use undetected chrome driver
02:48
47
047 Exploring Supabase for Efficient Database Management
06:32
48
048 Where to host the scrapers_ Render.com
04:11
49
049 Quick note on Concurrency in Web Scraping
02:44
50
050 How to make money web scraping
08:48
51
051 How to get web scraping clients
35:52
52
052 What do you charge clients
10:04
53
053 What should you scrape for
12:42
Unlock unlimited learning

Get instant access to all 52 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

What courses are similar to The Ultimate Web Scraping Course?

Frequently asked questions

What prerequisites are needed before taking this course?
The course does not explicitly list prerequisites, but a basic understanding of programming and web technologies would be beneficial. Familiarity with JavaScript, as used with Puppeteer, and concepts like HTML, APIs, and proxies may enhance learning, given the course covers topics like scraping HTML pages and using Puppeteer on AWS Lambda.
What kind of projects will I work on during the course?
Throughout the course, you will work on a variety of scraping projects, including scraping data from websites like Amazon, LinkedIn, and Instagram. You'll also learn to scrape Next.js sites and mobile apps, reverse engineer APIs, and employ Puppeteer for tasks like overcoming CAPTCHA challenges and retrieving TikTok videos.
What specific tools and platforms are covered in the course?
The course includes detailed lessons on using Puppeteer, AWS Lambda, and Render.com for web scraping tasks. You will also learn about proxies, using Google's Web Cache, and solving CAPTCHA challenges with Puppeteer. Furthermore, the course discusses leveraging Supabase for database management.
Who is the target audience for this course?
This course is aimed at individuals looking to learn web scraping for personal projects or as a career path. It's also suitable for those interested in finding clients and monetizing web scraping skills, as it includes sections on making money from web scraping and acquiring clients.
How does the scope of this course compare to others?
This course offers a comprehensive approach to web scraping, covering legal aspects, practical scraping techniques, and business-oriented topics like finding clients and growing on social media. Unlike some courses focused solely on the technical side, this one includes lessons on monetization and practical applications.
What topics are not covered in this course?
The course does not cover advanced programming concepts or deep dives into specific programming languages outside of those relevant to web scraping. It also doesn't focus on other data acquisition methods like data mining or focus on ethical hacking beyond differentiating it from scraping.
How much time should I expect to commit to this course?
While the total runtime of the course is not provided, given the 53 lessons, a reasonable estimate would be to allocate several weeks to complete the course if studying part-time. This timeline can vary based on prior knowledge and the time spent on practical exercises and projects.