Apache NiFi - A Complete Introduction | Master the Flow

Spin up NiFi on Windows by extracting the NiFi folder, setting data and logs location, running start and status scripts, and accessing the UI with generated credentials.

Explore the NiFi UI canvas to drag and connect processors, ports, and templates, and use the navigate and operate tabs to configure, run, save templates, and view data provenance.

Explore the NiFi user interface by creating a test process group with input and output ports, wiring get file and put file processors, and using templates.

Master how the NiFi expression language references flow file attributes and metadata with ${...}, and apply functions such as to upper, format, and substring to drive processor decisions.

Build a NiFi data flow that ingests files, sorts them by type, and routes music and text files into dedicated folders using get file, route on attribute, and put file.

Start the file processor to ingest input files into the data flow, then route audio and text files to their respective music files and text files directories.

Refactor sorting files in Apache NiFi by using update attribute to set a final file path, route music and text files, and write attributes before putting files.

Learn to run a NiFi data flow that sorts music and text files, and uses the update attribute processor to write a final file path attribute into flow files.

Explore NiFi properties and settings for processors, connections, and relationships, focusing on the get file processor creating flow files, input directory, attributes, and how connections manage priorities and scheduling.

Import csv data with get file, split records with split record, and configure csv reader and writer controller services for header-based schema to sort students by grade pass or fail.

Explore splitting a CSV with NiFi using the split record and extract text processors to derive grades as flow file attributes, tracking fragments with UUID and fragment index.

Learn to extract the grade from records using NiFi's extract text processor and a regex that captures the value after the last comma, writing it as a grade attribute.

Split a CSV into records, extract the grade, route failed and passing grades using root on attribute and expression language, then merge each group with the merge record processor.

Update the file name attribute to reflect failed or passing grades, then route the merged records to a put file processor that writes the CSV files to a local directory.

Learn to extract zipped data in Apache NiFi by ingesting with the get file processor, unpacking with the unpack content processor, and counting lines while logging attributes.

Edit a NiFi flow to compress extracted files using merge content with the D fragment strategy, rename the result to a zip, and write it to the output directory.

Generate flow files every five seconds for simulation using a generate flow file processor. Duplicate them with a copy count and detect duplicates with a detect duplicate processor, while logging.

Learn to set up a distributed map cache server in Apache NiFi, configure the client and server services, and detect duplicates by file name with logging and routing.

Master how to build a closed file loop in NiFi using wait and notify processors, fetching and putting files while logging attributes and leveraging a distributed cash service.

Apply the wait/notify pattern in Apache NiFi by using an update attribute processor to store the time in a cache attribute and leverage a distributed map cache for looped processing.

In Apache NiFi, build process groups to encapsulate count text and log attribute processors, create templates, and distribute load for batch flow files while handling back pressure.

Distribute flow files with distribute load processor to parallelize text processing, using five relationships and a round-robin strategy for bursts, and apply back pressure thresholds and funnels to merge results.