CF

Learn Web Scraping with NodeJs in 2020 - The Crash Course

7h 9m 10s
English
Paid

Master Web Scraping with NodeJs and tools such as Puppeteer (by Google), Request, Cheerio, and NightmareJs. Delve into the world of Web Scraping and Data Mining with NodeJs. Learn contemporary scraping methods using Puppeteer and direct NodeJs requests.

Course Highlights

Embark on a journey to enhance your web scraping skills. This course provides you with modern techniques to scrape websites using NodeJs and its popular libraries.

What You Will Learn

  • Create data scrapers from scratch using NodeJs
  • Identify and select the right tools for scraping various websites
  • Leverage top scraping tools for NodeJs to your advantage
  • Automate user interactions with NodeJs
  • Develop scrapers using Puppeteer by Google
  • Build scrapers utilizing the native Request & Cheerio libraries
  • Explore scraping techniques with NightmareJs

Requirements

  • Proficiency in JavaScript with ES6 syntax
  • Familiarity with CSS and jQuery selectors

Who Should Enroll

  • Complete beginners interested in learning data scraping with NodeJs
  • Individuals with basic knowledge seeking to elevate their skills

Course Structure and Benefits

This comprehensive crash course is designed for ease of understanding, catering to both absolute beginners and those with prior knowledge in the field.

  • Hands-on experience by working alongside the instructor with full access to all files and code samples
  • In-depth insights into problem-solving and best practices for building efficient scrapers
  • Comprehensive coverage that prepares you to confidently create your own web scrapers by the course's end

Course Overview

This course is not a trivial tutorial found online with basic examples. Instead, it dives into everything you need to consider when initiating the development of a web scraper with NodeJs. It includes problem-solving strategies and knowledge necessary to empower you to build your own scraper confidently.

Engage with real-world examples and tackle authentic challenges while learning how to build scrapers using NodeJs.

About the Author: Udemy

Udemy thumbnail

Udemy is the largest open marketplace for online courses on the internet. Founded in 2010 by Eren Bali, Oktay Caglar, and Gagan Biyani and headquartered in San Francisco, the company went public on the Nasdaq in 2021 under the ticker UDMY. The platform hosts well over two hundred thousand courses across software development, IT and cloud, data science, design, business, marketing, and creative skills, taught by tens of thousands of independent instructors. Roughly seventy million learners use it worldwide, and the corporate arm — Udemy Business — supplies a curated subset of that catalog to enterprise customers.

Because Udemy is a marketplace rather than a single editorial publisher, the catalog is uneven by design. The strongest material lives in the long-form, project-based courses authored by working engineers — full-stack JavaScript, React, Node.js, Python data science, AWS, Docker and Kubernetes, mobile development with Flutter and React Native, and cloud certification preparation. The CourseFlix listing under this source is the slice of that catalog that has been mirrored here for offline-friendly viewing, organized by topic and updated as new releases land. Pricing on Udemy itself swings dramatically with the site's near-permanent sales, which is why the platform is best treated as a deep reference catalog: pick instructors with strong reviews and a track record of updating their material rather than buying on the headline price alone.

Watch Online 50 lessons

This is a demo lesson (10:00 remaining)

You can watch up to 10 minutes for free. Subscribe to unlock all 50 lessons in this course and access 10,000+ hours of premium content across all courses.

View Pricing
0:00
/
#1: Introduction
All Course Lessons (50)
#Lesson TitleDurationAccess
1
Introduction Demo
02:12
2
Tools & Project Setup
03:30
3
Writing a Simple IMDB Scraper
10:32
4
Why & When to Choose Scraping
03:57
5
The biggest "problem" with scraping
02:38
6
Running with Terminal / CMD & Basics on VSCode Debugger
06:39
7
The Request library / method
05:13
8
The Browser Automation method
06:11
9
01 - How to spoof / add custom request headers
06:52
10
02 - Deal with GZIP Compression enabled websites
04:45
11
03 - Selectors & More Parsing
19:07
12
04 - Multiple Movies & Saving JSON to File
08:00
13
05 - Exporting data to CSV
06:49
14
06 - Easiest method of getting the proper selectors
05:37
15
07 - Downloading Images Locally
10:22
16
08 - Promisifying callback based functions
09:13
17
09 - Exposing and working with Custom Functions in Puppeteer
09:26
18
Using a Proxy Server
04:44
19
Status Codes & Error Handling
10:05
20
Manual Login with CSRF Protection & Cookie Parsing
18:36
21
01 - Introduction and inspecting the structures
08:29
22
02 - Building the scraper
20:25
23
03 - Parsing latest posts of an ig user
12:10
24
01 - Installation, introduction and testing
04:20
25
02 - Writing a simple automated task
09:05
26
03 - Generating PDF's, Emulating Phone Views, Getting Title & Url of pages
07:37
27
04 - Using Puppeteer to login on Instagram
11:36
28
05 - Making Websites Load Much Faster with a simple trick
08:24
29
06 - HTTP Basic Authentication
03:47
30
07 - Ignoring HTTPS / SSL Errors & Changing the ViewPort
03:41
31
08 - Learning how to use Proxies
02:56
32
01 - How to login on Twitter
10:23
33
02 - Refactoring code and improving it
07:16
34
03 - How to post Tweets with Puppeteer
11:34
35
04 - Scraping Twitter User Details
13:57
36
05 - Scraping More User Details
15:46
37
06 - Scraping User Tweets
14:10
38
07 - Scraping More Tweet Details ( Likes, Replies..etc )
14:11
39
08 - Handling The Infinite Loader of the Tweets
08:29
40
01 - Creating the Module Structure
06:44
41
02 - Parsing Product Details
11:29
42
03 - More Details and Tricks on the Product Details page of Amazon
07:01
43
01 - Installation, introduction and testing
06:29
44
02 - Talking and testing more about NightmareJs
07:16
45
03 - Scraping Ycombinator News with Vanilla Javascript
13:03
46
04 - Scraping Ycombinator News - More details
10:17
47
05 - Scraping Ycombinator News - Modulate & Pagination
11:41
48
06 - Handling Downloads
06:07
49
07 - Injecting Custom Javascript
05:24
50
Congratulations!
00:55
Unlock unlimited learning

Get instant access to all 49 lessons in this course, plus thousands of other premium courses. One subscription, unlimited knowledge.

Learn more about subscription

Related courses

Frequently asked questions

What prerequisites do I need before enrolling in this course?
Before enrolling, you should have proficiency in JavaScript, specifically with ES6 syntax. Additionally, familiarity with CSS and jQuery selectors is required. These skills are essential as the course involves using these technologies when building web scrapers and selecting elements on web pages.
What projects will I build during the course?
The course includes several hands-on projects such as creating a simple IMDB scraper, scraping Twitter user details, and building a scraper for Instagram posts. You will also work on scraping product details from Amazon and news from Ycombinator using various tools like Puppeteer, Request, and Cheerio.
Who is the target audience for this course?
The course is designed for both complete beginners who are interested in learning data scraping with NodeJs and individuals who already have basic knowledge and are looking to elevate their skills in this field.
How does the depth and scope of this course compare to other web scraping courses?
This course offers a practical and modern approach to web scraping, focusing on tools such as Puppeteer, Request, and Cheerio. It provides 50 lessons that cover a range of topics from basic scraping techniques to more complex automation tasks, making it suitable for beginners as well as those with some prior experience.
Which specific tools and libraries are covered in the course?
The course covers several popular NodeJs libraries and tools for web scraping, including Puppeteer by Google, the Request library, Cheerio, and NightmareJs. These tools are used to automate user interactions and scrape data efficiently from various websites.
What topics are not covered in this course?
The course does not cover advanced data analysis or machine learning techniques that might be used to process or analyze the data obtained from web scraping. It focuses strictly on the methods and tools necessary for scraping and does not delve into post-scraping data processing.
What is the expected time commitment for completing the course?
The course comprises 50 lessons, although the total runtime is not specified. It is designed for a comprehensive learning experience, providing hands-on projects and in-depth insights. Students should expect to invest a significant amount of time to fully engage with the materials and exercises provided.