The Ultimate Web Scraping Course

10h 33m 19s
English
Paid

I have been making a living with web scraping for over 3 years, earned more than $125,000, I have over 25,000 followers on social media, and in this course, I will tell exactly how I scrape websites, as well as how to find clients and grow on social media.

Read more about the course

In this course, I will show you how to scrape the following websites (and provide you with the code):

  • Instagram
  • Twitter
  • TikTok
  • LinkedIn
  • Zillow
  • Google Search
  • Google Maps
  • Amazon
  • Indeed
  • ESPN
  • Angi List
  • And more!

I will also explain how I found clients and gained over 25,000 followers on social media (and how you can do the same).

What's inside the course:

  • 9 hours of video materials
  • The legality of web scraping: what do you need to know?
  • Scrapers vs. hackers: what's the difference?
  • All the services I use (proxies, programming languages, servers, databases, packages for bypassing blocks)
  • How websites get their data (how APIs work)
  • My process for scraping ANY website
  • How to find the data source
  • How to avoid being blocked
  • All about proxies
  • Scraping by web frameworks (React, Next.js, static HTML/server-side rendering, ASP.NET, Java)
  • How to reverse engineer and bypass websites with good protection
  • Scraping websites with cursor-based pagination (e.g., comments on YouTube)
  • Scraping Chrome extensions to get free email addresses
  • How to scrape mobile apps
  • How to search for information about people (finding phone numbers and email addresses)
  • How to run Puppeteer on AWS Lambda
  • How to intercept requests with Puppeteer
  • Solving captchas with Puppeteer
  • How to find clients for web scraping
  • How to invoice clients
  • Product ideas based on web scraping

Disclaimers:

  • The code is written in JavaScript, and I use a Mac with VS Code.
  • I only briefly mention Puppeteer (there are much BETTER ways).
  • Web scraping is not magic. You need a source to scrape from (a website). For example, you cannot scrape people's bank accounts or something similar. That would fall under hacking, which is a different field.

Requirements:

  • You should have a good understanding of JavaScript or some other programming language to get the most out of this course.
  • If you cannot program at all, this course is not for you.
  • I do not teach the basics of programming or JavaScript in this course.

Watch Online The Ultimate Web Scraping Course

Join premium to watch
Go to premium
# Title Duration
1 001 The Ultimate Web Scraping Course 2024 - Google Docs - 30 May 2024 05:27
2 002 Legality of web scraping 08:37
3 003 Scraping vs Hacking 04:35
4 004 Services we'll be using in the course 04:36
5 005 Services we'll be using in the course 2 02:44
6 006 How websites get their data 07:58
7 007 My process for Scraping Any Site 04:16
8 008 How to find where the data is coming from 04:18
9 009 Proxies 21:23
10 010 Scraping Next.js sites (angi, imdb) 21:17
11 011 ESPN - Scraping API's 16:50
12 012 Amazon - Straight up HTML page 10:36
13 013 Update on Overcoming CloudFlare Blocks 04:23
14 014 Indeed - Straight Up HTML 25:14
15 015 Sarpy - ASP.NET (long version) 46:08
16 016 Sarpy - ASP.NET Cliff Notes 16:15
17 017 Kitsap - Java Site 22:07
18 018 Google Maps 46:16
19 019 Linkedin 08:55
20 020 How to scrape Shopify 07:50
21 021 They can't hide anything from you.... 01:52
22 022 Reverse Engineering Realtor API 27:52
23 023 Lynchburg exposing Auth Token 04:03
24 024 Beacons.ai 19:45
25 025 Twitter 18:29
26 026 Instagram Scraping Update 07:38
27 027 How to Scrape Instagram Profiles 03:42
28 028 Instagram 32:37
29 029 Arcgis 16:44
30 030 Cursor Based Pagination 11:54
31 031 Scraping Mobile Apps 09:34
32 032 Scraping Zillow Data Tutorial 11:33
33 033 Skip Tracing 09:39
34 034 Getting Puppeteer Up and Running on AWS Lambda 15:39
35 035 facbleag 08:01
36 036 Getting Tiktok Videos with Puppeteer 05:15
37 037 Solving CAPTCHAs with Puppeteer 09:44
38 038 Sitemaps (crunchbase) 10:29
39 039 Update 12312024 Use puppeteer to get company pages 04:59
40 040 Google Tricks and Tips 03:49
41 041 Decoding Imperva Security Measures 02:49
42 042 Leveraging Google's Web Cache to get around Cloudflare and Datadome blocking 03:35
43 043 Scraping Kick (got-scraping + use your resources) 08:21
44 044 I found a way to scrape Ohio State Student Emails 07:15
45 045 Tiktok API 04:35
46 046 SC Courts, use undetected chrome driver 02:48
47 047 Exploring Supabase for Efficient Database Management 06:32
48 048 Where to host the scrapers_ Render.com 04:11
49 049 Quick note on Concurrency in Web Scraping 02:44
50 050 How to make money web scraping 08:48
51 051 How to get web scraping clients 35:52
52 052 What do you charge clients 10:04
53 053 What should you scrape for 12:42

Similar courses to The Ultimate Web Scraping Course

Web Components For Beginners

Web Components For Beginnersleveluptutorials

Category: Other (Frontend)
Duration 3 hours 1 minute 52 seconds
Youtube clone

Youtube cloneNomad Coders

Category: JavaScript, Node.js, MongoDB
Duration 15 hours 41 minutes 20 seconds
Vanilla Three.js Course

Vanilla Three.js Coursejsmastery.proAdrian Hajdin

Category: Other (Frontend)
Duration 31 minutes 46 seconds
Become a WordPress Developer: Unlocking Power With Code

Become a WordPress Developer: Unlocking Power With Codeudemy

Category: JavaScript, PHP, Wordpress
Duration 41 hours 45 minutes 55 seconds
Shaders for the Web

Shaders for the Websuperhi.com

Category: JavaScript, Three.js, OpenGL Shading Language (GLSL)
Duration 16 hours 3 minutes 4 seconds
Machine Learning in JavaScript with TensorFlow.js

Machine Learning in JavaScript with TensorFlow.jsudemy

Category: JavaScript, Data processing and analysis
Duration 6 hours 42 minutes 20 seconds