The Ultimate Web Scraping Course

10h 33m 19s
English
Paid

Course description

I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow on social media.

Read more about the course

In this course, I will show you how to scrape the following websites (and provide you with the code):

  • Instagram
  • Twitter
  • TikTok
  • LinkedIn
  • Zillow
  • Google Search
  • Google Maps
  • Amazon
  • Indeed
  • ESPN
  • Angi List
  • And more!

I will also explain how I found clients and gained over 25,000 followers on social media (and how you can do the same).

What's inside the course:

  • 9 hours of video materials
  • The legality of web scraping: what do you need to know?
  • Scrapers vs. hackers: what's the difference?
  • All the services I use (proxies, programming languages, servers, databases, packages for bypassing blocks)
  • How websites get their data (how APIs work)
  • My process for scraping ANY website
  • How to find the data source
  • How to avoid being blocked
  • All about proxies
  • Scraping by web frameworks (React, Next.js, static HTML/server-side rendering, ASP.NET, Java)
  • How to reverse engineer and bypass websites with good protection
  • Scraping websites with cursor-based pagination (e.g., comments on YouTube)
  • Scraping Chrome extensions to get free email addresses
  • How to scrape mobile apps
  • How to search for information about people (finding phone numbers and email addresses)
  • How to run Puppeteer on AWS Lambda
  • How to intercept requests with Puppeteer
  • Solving captchas with Puppeteer
  • How to find clients for web scraping
  • How to invoice clients
  • Product ideas based on web scraping

Disclaimers:

  • The code is written in JavaScript, and I use a Mac with VS Code.
  • I only briefly mention Puppeteer (there are much BETTER ways).
  • Web scraping is not magic. You need a source to scrape from (a website). For example, you cannot scrape people's bank accounts or something similar. That would fall under hacking, which is a different field.

Requirements:

  • You should have a good understanding of JavaScript or some other programming language to get the most out of this course.
  • If you cannot program at all, this course is not for you.
  • I do not teach the basics of programming or JavaScript in this course.

Watch Online

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 53 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: 001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024

All Course Lessons (53)

#Lesson TitleDurationAccess
1
001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024 Demo
05:27
2
002 Legality of web scraping
08:37
3
003 Scraping vs Hacking
04:35
4
004 Services we'll be using in the course
04:36
5
005 Services we'll be using in the course 2
02:44
6
006 How websites get their data
07:58
7
007 My process for Scraping Any Site
04:16
8
008 How to find where the data is coming from
04:18
9
009 Proxies
21:23
10
010 Scraping Next.js sites (angi, imdb)
21:17
11
011 ESPN - Scraping API's
16:50
12
012 Amazon - Straight up HTML page
10:36
13
013 Update on Overcoming CloudFlare Blocks
04:23
14
014 Indeed - Straight Up HTML
25:14
15
015 Sarpy - ASP.NET (long version)
46:08
16
016 Sarpy - ASP.NET Cliff Notes
16:15
17
017 Kitsap - Java Site
22:07
18
018 Google Maps
46:16
19
019 Linkedin
08:55
20
020 How to scrape Shopify
07:50
21
021 They can't hide anything from you....
01:52
22
022 Reverse Engineering Realtor API
27:52
23
023 Lynchburg exposing Auth Token
04:03
24
024 Beacons.ai
19:45
25
025 Twitter
18:29
26
026 Instagram Scraping Update
07:38
27
027 How to Scrape Instagram Profiles
03:42
28
028 Instagram
32:37
29
029 Arcgis
16:44
30
030 Cursor Based Pagination
11:54
31
031 Scraping Mobile Apps
09:34
32
032 Scraping Zillow Data Tutorial
11:33
33
033 Skip Tracing
09:39
34
034 Getting Puppeteer Up and Running on AWS Lambda
15:39
35
035 facbleag
08:01
36
036 Getting Tiktok Videos with Puppeteer
05:15
37
037 Solving CAPTCHAs with Puppeteer
09:44
38
038 Sitemaps (crunchbase)
10:29
39
039 Update 12312024 Use puppeteer to get company pages
04:59
40
040 Google Tricks and Tips
03:49
41
041 Decoding Imperva Security Measures
02:49
42
042 Leveraging Google's Web Cache to get around Cloudflare and Datadome blocking
03:35
43
043 Scraping Kick (got-scraping + use your resources)
08:21
44
044 I found a way to scrape Ohio State Student Emails
07:15
45
045 Tiktok API
04:35
46
046 SC Courts, use undetected chrome driver
02:48
47
047 Exploring Supabase for Efficient Database Management
06:32
48
048 Where to host the scrapers_ Render.com
04:11
49
049 Quick note on Concurrency in Web Scraping
02:44
50
050 How to make money web scraping
08:48
51
051 How to get web scraping clients
35:52
52
052 What do you charge clients
10:04
53
053 What should you scrape for
12:42

Unlock unlimited learning

Get instant access to all 52 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Comments

0 comments

Want to join the conversation?

Sign in to comment

Similar courses

Modern JavaScript From The Beginning 2.0

Modern JavaScript From The Beginning 2.0

Sources: Brad Traversy
This is a 37+ hour in-depth course that will take you from the absolute beginning of JavaScript, learning about data types, functions and loops to learning DOM
36 hours 42 minutes 18 seconds
20 Web Projects With Vanilla JavaScript

20 Web Projects With Vanilla JavaScript

Sources: udemy, Brad Traversy
This is a fun, practical & project based course for all skill levels. The projects in this course are designed to get you building things using HTML5, CSS & JavaScript with no ...
16 hours 8 minutes 55 seconds
70+ JavaScript Challenges: Data Structures & Algorithms

70+ JavaScript Challenges: Data Structures & Algorithms

Sources: Brad Traversy
Master JavaScript challenges with a focus on data structures and algorithms. Improve problem-solving skills, learn recursion, and tackle coding interviews
12 hours 29 minutes 29 seconds
Advanced JavaScript Unleashed

Advanced JavaScript Unleashed

Sources: fullstack.io
JavaScript is arguably the most widely used programming language on the planet, and there is a vast amount of content available for learning JavaScript. The problem is that not ...