Udemy
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
  •  
Development
Web Development Data Science Mobile Development Programming Languages Game Development Database Design & Development Software Testing Software Engineering Development Tools No-Code Development
Business
Entrepreneurship Communications Management Sales Business Strategy Operations Project Management Business Law Business Analytics & Intelligence Human Resources Industry E-Commerce Media Real Estate Other Business
Finance & Accounting
Accounting & Bookkeeping Compliance Cryptocurrency & Blockchain Economics Finance Finance Cert & Exam Prep Financial Modeling & Analysis Investing & Trading Money Management Tools Taxes Other Finance & Accounting
IT & Software
IT Certification Network & Security Hardware Operating Systems Other IT & Software
Office Productivity
Microsoft Apple Google SAP Oracle Other Office Productivity
Personal Development
Personal Transformation Personal Productivity Leadership Career Development Parenting & Relationships Happiness Esoteric Practices Religion & Spirituality Personal Brand Building Creativity Influence Self Esteem & Confidence Stress Management Memory & Study Skills Motivation Other Personal Development
Design
Web Design Graphic Design & Illustration Design Tools User Experience Design Game Design Design Thinking 3D & Animation Fashion Design Architectural Design Interior Design Other Design
Marketing
Digital Marketing Search Engine Optimization Social Media Marketing Branding Marketing Fundamentals Marketing Analytics & Automation Public Relations Advertising Video & Mobile Marketing Content Marketing Growth Hacking Affiliate Marketing Product Marketing Other Marketing
Lifestyle
Arts & Crafts Beauty & Makeup Esoteric Practices Food & Beverage Gaming Home Improvement Pet Care & Training Travel Other Lifestyle
Photography & Video
Digital Photography Photography Portrait Photography Photography Tools Commercial Photography Video Design Other Photography & Video
Health & Fitness
Fitness General Health Sports Nutrition Yoga Mental Health Dieting Self Defense Safety & First Aid Dance Meditation Other Health & Fitness
Music
Instruments Music Production Music Fundamentals Vocal Music Techniques Music Software Other Music
Teaching & Academics
Engineering Humanities Math Science Online Education Social Science Language Teacher Training Test Prep Other Teaching & Academics
AWS Certification Microsoft Certification AWS Certified Solutions Architect - Associate AWS Certified Cloud Practitioner CompTIA A+ Cisco CCNA Amazon AWS CompTIA Security+ AWS Certified Developer - Associate
Graphic Design Photoshop Adobe Illustrator Drawing Digital Painting InDesign Character Design Canva Figure Drawing
Life Coach Training Neuro-Linguistic Programming Personal Development Mindfulness Meditation Personal Transformation Life Purpose Emotional Intelligence Neuroscience
Web Development JavaScript React CSS Angular PHP WordPress Node.Js Python
Google Flutter Android Development iOS Development Swift React Native Dart Programming Language Mobile Development Kotlin SwiftUI
Digital Marketing Google Ads (Adwords) Social Media Marketing Google Ads (AdWords) Certification Marketing Strategy Internet Marketing YouTube Marketing Email Marketing Google Analytics
SQL Microsoft Power BI Tableau Business Analysis Business Intelligence MySQL Data Modeling Data Analysis Big Data
Business Fundamentals Entrepreneurship Fundamentals Business Strategy Online Business Business Plan Startup Blogging Freelancing Home Business
Unity Game Development Fundamentals Unreal Engine C# 3D Game Development C++ 2D Game Development Unreal Engine Blueprints Blender
30-Day Money-Back Guarantee
Development Web Development Java

Professional Web Scraping with Java

Learn how to scrape data from any static or dynamic / AJAX web page using Java in a short and concise way.
Rating: 4.1 out of 54.1 (163 ratings)
1,265 students
Created by Patrick Meier
Last updated 8/2018
English
English [Auto]
30-Day Money-Back Guarantee

What you'll learn

  • Have a solid understanding of web scraping with Java
  • Beeing able to scrape practically any web page (static AND dynamic / AJAX) though you learn the concepts behind web scraping
  • Download, parse and extract data from websites with Jsoup
  • Call web APIs in Java with Unirest
  • Export your data as CSV or JSON
  • Build web scrapers that stay undetected and do not get blocked or banned

Course content

6 sections • 18 lectures • 1h 23m total length

  • Preview01:53

  • Preview00:52
  • Concept: how to scrape static web pages
    02:00
  • Preview05:45
  • Example: scraping the top 10 Google search results
    14:03

  • Preview01:55
  • Concept: how to scrape dynamic / AJAX web pages
    02:35
  • Making HTTP Requests with Unirest
    11:20
  • Example: scraping results from peoplefinders.com
    14:59
  • Bonus: Proxycrawl to scrape dynamic pages
    00:17
  • Dealing with WebSockets
    06:39

  • Export your data as CSV
    02:10
  • Export your data as JSON
    04:22

  • How to stay undetected
    02:22
  • Bonus: Overview of Proxycrawl
    05:37
  • Bonus: Example using Proxycrawl for peoplefinders.com
    05:07

  • Conclusion and what's next
    01:20
  • Bonus: Mind Map of the course content
    00:07

Requirements

  • You should already be familiar with Java and Maven at a basic to medium level (the course will not show you how to setup Java, Maven or an IDE)
  • You should be familiar with HTML/CSS and know how to use your browser's developer tools
  • You should know about CSS selectors, though we use them for scraping static web pages
  • Prior knowledge of jQuery helps you getting started faster with Jsoup, though this is not required
  • You should know what a web API and AJAX is (basic level is enough)

Description

In this short and concise course you will learn everything to get started with web scraping using Java. 

You will learn the concepts behind web scraping that you can apply to practically any web page (static AND dynamic / AJAX).


Course structure

We start with an overview of what web scraping is and what you can do with it. 

Then we explain the difference in scraping static pages vs dynamic / AJAX pages. You learn how to classify a website in one of the two categories and then apply the right concept in order to scrape the data you want.

Now you will learn how to export the scraped data either as CSV or JSON. These are some popular formats that can be used for further processing.

Unfortunately many websites try to block scrapers or sometimes you just do not want to be detected. In the section going undercover you will learn how to stay undetected and avoid getting blocked.

At the end of the course you can download the full source code of all the lectures and we discuss an outlook to some advanced topics (private proxies, cloud deployment, multi threading ...). Those advanced topics are covered in a follow up course I am going to teach.


Why you should take this course

Stop imagining you can scrape data from websites and use the skills for your next web project, you can do it now.

  • Stay ahead of your competition
  • Be more efficient and automate tedious, manual tasks
  • Increase your value by adding web scraping to your skill set


Enroll now!

Who this course is for:

  • Anyone with an interest in learning web scraping and understanding the concepts
  • Anyone who likes a short and concise course
  • This course is NOT an introduction to Java
  • This course will NOT show you how to setup your development environment
  • This course is intended to get you started with web scraping. Very advanced topics (e.g. private proxies, cloud deployment, multi threading) are discussed but not implemented in this course. I will do an an advanced / enterprise level course on this separately...
  • Windows, Mac, or Linux PC

Instructor

Patrick Meier
Entrepreneur, Software Developer
Patrick Meier
  • 4.1 Instructor Rating
  • 163 Reviews
  • 2,847 Students
  • 1 Course

I am an entrepreneur and software developer who really enjoys to build and learn new things. I now have over 9 years of experience from working in different companies (big and small) and even founding my own startup .

I built several scalable backend systems in the cloud running on Java and Spring. Then I discovered JavaScript as a language for creating different kinds of things - from webapps to mobile apps and even the backend using NodeJs.

I love to share what I have learned with YOU to be more effective and successful.

  • Udemy for Business
  • Teach on Udemy
  • Get the app
  • About us
  • Contact us
  • Careers
  • Blog
  • Help and Support
  • Affiliate
  • Impressum Kontakt
  • Terms
  • Privacy policy
  • Cookie settings
  • Sitemap
  • Featured courses
Udemy
© 2021 Udemy, Inc.