Web Scraping on Java 17 & HtmlUnit

This is a "Follow me" product development course.

In this project you will touch on:

• Java 8+, lambda expressions, var

• Maven - modification, assembly of the jar archive and its subsequent launch

• XPath - and its syntax

• Testing

• Multithreading

• Applications of Singleton and Factory patterns

• working with regular expressions

• working with Gson and saving a project

And we won't use Spring just out of spite)

The project begins with Intelliji IDEA Community installation, deploying the base project on Maven and till the jar build so that the program can work independently without an IDE (development environment). The parsing result will written as a json files to a hostnamed forlder.

If you understand at least some of the words and if you are interested in your own website parsing project, this course is for you.

I set the entry level for this course as an intermediate level and at the same time kept the entry level in mind. At the end, the boundaries between entry-level and intermediate levels are largely blurred, and a person considering his level to be average and with experience in a company can “skip” some initial knowledge and vice versa.

In this course, I apply work experience training new developers, where we work through a task together, conveying not only an understanding of the design, but also the design approach and semantics.

Although the training is built on the principle of do as I'm, I still think it is important to explain what is happening on the screen. If it's not enough, I'll add it.

Let's get started.