Batch Processing with Spring Batch & Spring Boot

Download and install JDK 8, Apache Maven, and the Spring Tool Suite on your local machine, then configure Javahome and M2 Home and update your path.

Install postman as the REST client from its official website, download for your OS, open the app, and learn to send requests with query parameters and a JSON body.

Download and install MySQL on your local machine, including the server and MySQL Workbench, start the server, and connect to localhost:3306 with MySQL Workbench to support your Spring Boot application.

Explore batch processing as bulk handling of large data, not in real time, with banking and e-commerce examples where transactions or updates are collected and processed later by batch jobs.

Discover how Spring Batch enables high-volume batch processing within Spring Boot, reading from CSV, JSON, or XML and writing to databases like MySQL with easy configuration.

Understand how Spring Batch uses jobs and steps for batch processing, showing that a job can include multiple steps or a single step in a banking example.

Explore the two spring batch step types: tasklet and chunk-oriented. See how chunk size governs batch processing with csv to mysql examples, using item reader, processor, and writer.

Understand the spring batch architecture, where jobs consist of single or multiple steps, including chunk oriented steps with reader, processor, and writer; learn how job repository and launcher manage metadata.

Create your first spring batch application by initializing a spring boot project, adding spring batch and H2 in-memory database dependencies, and enabling batch processing in the main class.

Create your first Spring Batch job with a single tasklet step in Spring Boot, using job builder factory and step builder factory to run a simple task and finish.

Add a second tasklet step to a Spring Batch job using the step builder factory, define its tasklet, and orchestrate steps with start and next to ensure ordered execution.

Customize a Spring Batch Tasklet step by moving logic into a class that implements Tasklet, then autowire it and note the need for an SQL database for job repository.

Learn how a job instance handles daily executions, how a job execution captures start, end, and status, and how the job execution context stores key-value data for steps.

Explore step execution context and job execution context in spring batch, covering step level contexts. Understand how each step has execution and status, maps stored in the database.

Configure spring batch with a MySQL database by creating a spring_batch schema, updating dependencies, and enabling schema initialization to store job and step metadata.

Configure Spring Batch with a MySQL schema, run the first job to create tables, and review how job instance, job execution, and step execution metadata are stored.

Pass unique job parameters to spring batch to create a new job instance and execute steps, using keys like run to distinguish runs and avoid reusing completed instances.

Apply the run id incrementer to guarantee unique job parameters in spring batch. Let the incrementer increment the run.id parameter for each job execution in spring batch with spring boot.

Implement a Spring Batch job execution listener to intercept before and after job runs, access job parameters and execution context, and propagate context to steps.

Learn how to intercept step execution in Spring Batch with a step execution listener, using before step and after step hooks and accessing step and job execution contexts.

Discover the chunk oriented step in spring batch with item reader, processor, and writer, using a basic example to configure multiple jobs in spring batch and spring boot.

Create the first item reader in Spring Batch and Spring Boot application, reading integers from a list one by one via the read method and signaling end by returning null.

Create your first item processor in Spring Batch by converting input integers to longs, adding 20, and bridging the reader to the writer.

Create your first item writer in Spring Batch, printing lists of long values whose size matches the configured chunk size and illustrating writer behavior after the item processor.

create a chunk oriented step by wiring a reader, processor, and writer with a chunk size of three, as part of batch processing with Spring Batch and Spring Boot.

Execute a chunk oriented step in Spring Batch with a chunk size of three, showing the item reader, processor, and writer and null signals end and read and write counts.

Experiment with a chunk oriented step in Spring Batch using only item reader and item writer, omitting the item processor, and adjust data types accordingly.

Combine tasklet and chunk oriented steps in a Spring Batch job, sequencing a chunk step and a tasklet step (or vice versa) using next, with a chunk size of four.

Explore how to control startup behavior of Spring Batch jobs by configuring spring.batch.job.enabled, enabling manual triggering via rest API and scheduling with Spring Scheduler for production batch processing.

Create a rest api to launch spring batch jobs with a spring boot web app, wiring a job launcher and two qualified jobs, passing current time milliseconds as parameters.

Use a rest gate API in Spring Boot to start Spring Batch jobs, then enable asynchronous execution so the API returns immediately while the job runs in the background.

Pass custom job parameters to a spring batch job via a gate api by sending a json array of key-value pairs and mapping them to the job parameters.

Enable scheduling in spring boot application and configure a scheduler to launch a batch job at intervals with a cron expression. Pass current time in milliseconds as a job parameter.

Stop a running Spring Batch job with a REST API using the JobOperator, passing the job execution ID, and learn how the current chunk completes before stopping.

Explore how Spring Batch and Spring Boot use diverse item readers to read from csv, json, xml, databases, and rest apis, enabling efficient chunk-oriented processing.

Explore configuring the Spring Batch flat file item reader to read a CSV file, including setting the CSV location, delimiter, headers, and a bean-mapped model class via the line mapper.

Learn to build a flat file item reader for a CSV file using Spring Batch, configuring CSV resource, line mapper, and header skipping to map rows to student CSV model.

Learn to read a CSV with a flat file item reader in Spring Batch, map fields with a fieldset mapper, and print results to the console.

Change the flat file item reader delimiter from comma to pipe, using either a parameterized constructor or the set delimiter method, and verify the read results in Spring Batch.

pass the csv file path as a job parameter and read it with a flat file item reader using a file system resource.

Explore customizing a flat file item reader by configuring a default line mapper with a delimited line tokenizer and a fieldset mapper, comparing inline and separate-object approaches.

Learn how to read a json file with the json item reader by defining a student model class with id, first name, last name, and email.

Create a json item reader using jackson to read studentJson with a job parameter for the input json file in spring batch.

Configure the Json item reader by passing the json file path as a job parameter, map keys with @JsonProperty, ignore unknowns with @JsonIgnoreProperties, and control reading with max item count.

Create a sample students.xml and a corresponding student model class to feed data via an xml item reader, with four fields and an xml root annotation.

Configure a spring batch XML item reader using Stax and unmarshalling. Add the OKM dependency in pom.xml, set the fragment root to student, and bind Jaxb2Marshaller to the StudentXML model.

Run the spring batch app with xml item reader, supply xml file path, print id, first name, last name, email for ten items, mapping mismatched attributes via xml element annotation.

Read the student table from a MySQL data source using Spring Batch Jdbc item reader, mapping each row to a student Jdbc model in the Spring Batch schema.

Configure a Jdbc cursor item reader to read from the MySQL student table, alias first and last name to match Student Jdbc model, and map rows with a row mapper.

Explore the JDBC item reader in a Spring Boot app, reading rows from the student table, and learn how aliasing, missing columns, and skip settings affect data mapping.

Configure multiple data sources for a spring batch app by separating metadata and data into two schemas, and wire distinct data source beans to the Jdbc item reader.

Learn to read from a rest api in spring batch via an item reader using rest template and a model class to convert json into a student list.

learn to read from a rest api using spring batch's item reader adapter, returning one student at a time via a student service method in a chunk-oriented step.

Run a rest api read in spring batch using the item reader adapter to call get student with id and name, processing ten items in chunks of three.

Explore the various item writers in Spring Batch, including CSV, JSON, XML, database, and REST API writers, and learn how to configure writing to different destinations.

Create a flat file item writer to write from MySQL student table to a CSV file, with headers, a delimited line aggregator, and a footer showing the created at date.

Set the output file path and run the job to generate a csv with headers and data from MySQL using a flat file item writer in Spring Batch.

Read from a MySQL database with a Jdbc item reader and write to a Json file using Spring Batch's Json item writer, marshalling Java objects with Jackson.

Use an item processor to map data from a jdbc item reader to a json item writer, with separate model classes for reading from MySQL and writing to json.

Explore XML item writer in Spring Batch, writing Java objects to an XML file using a marshaller, with a reader-writer model like student Jdbc and the root tag students.

Read from a csv file with a flat file item reader and write into a MySQL database using a Jdbc batch item writer with named parameters and a bean annotation.

Implement a JDBC item writer with a prepared statement to insert ten records into the MySQL student table from a CSV file, replacing named parameters with a prepared statement setter.

Learn to call a rest api from a spring batch item writer by posting CSV-derived student data via rest template in json.

Invoke a rest api call from the item writer using the item writer adapter to post student data read from a csv with spring batch.

Explore fault tolerance in Spring Batch and Spring Boot by skipping bad records and retrying when read or write operations fail to keep the job running.

Discover why fault tolerance matters in spring batch jobs, illustrated by a csv flat file pipeline that handles bad records without losing valid data.

Learn how to skip bad records in a fault-tolerant batch workflow by configuring skip limits, applying a skip policy, and managing exceptions in the reader, processor, and writer.

Capture bad records from the item reader in Spring Batch using a skip listener, writing them to per-step text files for easy troubleshooting.

Learn how to capture bad records during Spring Batch processing by using skip in process, a processor listener, and separate text files for problematic records, including handling null pointer exceptions.

Capture bad records while writing data by leveraging skip logic in the item writer and a skip listener, auditing offending records to identify issues in reader, processor, or writer.

Implement skip listener in Spring Batch to capture skipped records in batch jobs, wiring a class that handles read, process, and write skips to monitor bad data during batch processing.

Learn how Spring Batch retry mechanism lets you retry failed items in the item processor or item writer, not the item reader, addressing business layer issues instead of skipping records.

Explore configuring a retry mechanism in a junk oriented step, including retry limits, default zero, exception types, and the safe use of skip policies to avoid infinite loops.

migrate databases in real time using Spring Batch by reading from Postgres and writing to MySQL with dedicated item reader and writer, handling table relationships and data types.

Download and install PostgreSQL on your local machine, remember the password, open the admin UI, and create a blank university database for upcoming migration.

Import and restore a university database in PostgreSQL from a SQL file, inspect department, staff, and subject tables, and explore ER diagram and data type mismatches before migrating to MySQL.

Update the app to the latest Spring Boot 2.6, download dependencies, verify, and resolve a bean cycle by moving the job builder factory and data source to a database config.

Configure Spring Boot with Spring Data JPA to set up two data sources and migrate tables from Postgres university schema to MySQL using a GPI reader and writer.

Create JPA entity classes for Postgres and MySQL to map each table, using @Entity, @Table, and @Column with proper primary key, underscored column names, and database-specific data types.

Configure two entity manager factories for Postgres and MySQL, wire their data sources and entity packages, and integrate them with the GPA reader and writer.

Add the Postgres dependency and configure a JPA item reader in Spring Batch using an entity manager factory to read all records from the Postgres student table via JPA query.

Configure a JPA item writer in Spring Batch by specifying the entity class, wiring the entity manager factory for a MySQL database, and adding a primary transaction manager.

Learn to build Spring Batch JPA item reader and writer with an item processor that resolves boolean data type mismatches by converting nulls to false during 100-record Postgres migration.

Learn how to migrate data from Postgres to MySQL with Spring Batch, using a transaction-managed step and verification. The lecture covers parent-child table migration and a subject table assignment.

Explore dynamic batch migration with Spring Batch and Spring Boot by using job parameters and current item count and max item count to process records in chunks.