
Explore functional streams for Scala with Fs2, learning to create and compose pure and effect streams, perform extreme transformations, and coordinate concurrent streaming and communication with Cats and Cats Effect.
Set up a scala fs2 project by creating the source directory and an example app with a run method, then run it in IntelliJ. Use worksheets to test streams.
Demonstrates imperative CSV processing of lego set data using a buffer, parsing fields, and ignoring malformed lines, then compares with iterator and stream approaches in FS2 to show tradeoffs.
Extend an imperative csv processing algorithm by filtering with a predicate and enforcing a limit on results, using option's future and careful counter placement.
Process a csv file with functional streams by loading lines into a list, mapping and filtering optional values, and taking a maximum number of results.
Compare an imperative, high-performance csv reader with a readable iterator-based approach that streams lines on demand using source.fromFile, applies filter and take, and manages resources.
Learn to process a csv file with fs2 streams in scala, using the io monad for referential transparency, decoding bytes to utf-8 and mapping to lines, and filtering optionals.
Explain fs2 streaming of a csv file: read lines, parse, filter, and limit with explicit effect type, then switch to parallel map and throttle one line per second.
Explore finite and infinite streams in fs2, creating empty, single-value, and range streams; learn to iterate, fold, and constant streams, and combine with map, filter, and concatenation.
Learn to create finite streams with fs2 in Scala, using emit for single values, emits for sequences, including empty streams and vectors, and inspect content by converting to lists.
Explore lazy streams using iterate to generate an infinite stream of natural numbers, starting from one, applying a next value function, and safely inspect by taking the first five elements.
Show how unfold builds a stream from an initial state using an optional (output, next state) and stop signal, emitting numbers 1–5 or ending at five.
Demonstrate two fs2 stream techniques: range yields 1 to 14, and constant produces an infinite 42 stream, revealing how to take a finite sample.
Define a function that produces a stream using iterate. Take the initial value a and add one to each next element, then take 26 letters to yield a through z.
Demonstrates producing the alphabet as a stream using unfold only, by defining a state function that outputs characters from A to Z and stops after the last letter.
Implement iterate in terms of unfold using an initial value and a next value function to produce an infinite stream of states.
Concatenate streams with the ++ operator to combine simple and iterative streams, observe results with toList, and note left bias when an infinite left stream prevents the right from reaching.
Use the map method to double every element in a finite stream, then apply the same technique to an infinite stream, take a sample of ten elements, and illustrate evens.
This lecture demonstrates flatMap on pure streams, showing how mapping to i and i+1 flattens into a single stream or list, including infinite streams and lazy evaluation.
Filter a stream of natural numbers to keep those with modulo two equal to one; use for comprehension with map and yield to produce and convert results into a list.
Explore zip and zipWith on streams, including handling infinite streams and stopping when one input ends, while multiplying elements and pairing evens with natural numbers to produce samples.
Use fold on a finite fs2 stream, like the good old fold from collections, to compute a length or sum, producing a new stream that emits a single value.
Produce the odd numbers from an infinite stream using map, by applying 2*n - 1, and test with a sample list like 1, 3, 5, 7, 9.
Implement a repeat method on pure streams that repeats input elements indefinitely. Use a recursive approach to cycle through 1 2 3, producing 1 2 3, 1 2 3 indefinitely.
Implement the unNone method to remove None values from a stream of option elements, flattening Some values using stream from option and a for-comprehension.
Pure streams in FS2 are lazy, recipe-based collections you can create and combine. To realize elements, call toList, but beware infinite streams that never end.
Discover effectful streams with FS2 in Scala, lifting effects from pure streams, iterating with state, composing with map and flatMap, and managing resources, timing, and errors.
Explore effect filtering in FS2 by moving from pure streams to effectful streams, compiling to an IO and draining to run side effects without emitting values.
Demonstrates lifting an effect into a stream that emits no elements, showing a second effect stream that executes the effect and returns nothing, and how to run it.
Covary a pure stream into io to enable io-specific combinators for effectful streams in fs2. The example prints one, two, three to the console after covarying to io.
Explore the evolved iterate method, starting from an initial value and an effectful function, producing values with effects in an infinite stream and illustrating the Drake method.
This lecture explains unfolding in functional streams, showing how an option signals termination and a pair of value and state drives alphabet generation, with IO side effects wrapping the process.
practice implementing pagination for mock film streams in fs2, scala; fetch pages by number and size, build a single stream with unfold and evolve, stopping when a page is empty.
Explore functional streams in Scala with FS2 by building and running streams, inspecting values like 42, and using repeat to create infinite side-effecting streams with finite samples via take.
Explore for comprehensions in fs2 by composing streams with flat map, map, and filter, using pure streams and f type streams to yield values like 42 and x plus one.
Learn how to use evalMap, evalTap, and evalFilter in functional streams for scala with fs2 to perform effects, print elements, and filter streams with side effects.
Apply the plus plus operator to concatenate streams and sequence effects, then use snip to delay execution semantically for a duration and observe the output.
Implement eyeball every as a finite-duration sleep loop that executes an effect, emits a value, and repeats indefinitely. Take a finite sample to observe outputs from the infinite stream.
Explore error handling in FS2 streams, including race error and handle, and learn to recover and continue with rights while discarding lefts using an attempt-style syntax with flatMap.
Learn how to safely manage file readers as resources in FS2, using bracket and resource patterns to acquire, release, and stream lines with proper error handling.
Control timing in fs2 streams by using time primitives like timeout, interrupt after, and delay by to start, pause, or stop streams gracefully within a given duration.
Explore throttling using the time method in fs2, applying a meter with a rate like 1/2 to regulate emissions, and account for execution time to emit immediately or after waiting.
Explore debouncing in functional streams by modeling resize events with iterate and map, then apply the bounce sample window of 200 milliseconds and observe sampled outputs with tap.
Explore how to retry a failing effect in fs2 streams using a retry function with configurable delays, exponential backoff, and a maximum number of attempts.
Practice building and testing a streaming search experience using fs2, simulating user typing with incremental streams, handling intermittent failures, retries, sampling, and bounded runtime.
Review how functional streams produce values and effects, create streams from effect types IO, and return to IO with to list or drain while handling resources, timeouts, and retries.
Explore the fundamentals of functional streams by comparing pool-based and push-based models, learn why streams emit in chunks, and introduce pools and pipes as core transformative concepts.
Explore pull-based streams in fs2, where elements are produced on demand by the consumer, and push-based streams where the producer emits at its own pace to registered callbacks.
Explore the chunk structure of streams, comparing pure streams that emit a single chunk with effectful streams that produce multiple chunks, and how concatenation and file IO influence sizes.
Explore the chunk API by building chunks from elements, arrays, and singletons, and compare them with lists. Learn concatenation, indexing, and the compact method to create a single array-backed chunk.
Implement the compact method by converting any chunk to an array-backed chunk using copy to array and the array factory, with examples.
Discover how pools and chunks power pipes that transform streams with fs2, using pure outputs, chunked emissions, and flatMap-based composition to enable stateful transformations.
Turn a stream into a pool (pull) and examine how echo and take create a pool, including a rest-of-stream represented by option and the remaining elements.
Learn to implement a skipLimit function for streams, keeping a fixed number of elements after skipping a set, with practical examples like 1..100 and 1..4.
Explore a first chunk pipe in functional streams for Scala with FS2, extracting the first chunk from a stream, returning the rest, and composing with pipe syntax and monadic operations.
Explore building a drop pipe for fs2 to drop a set number of elements from an input stream, handling chunked data and emitting the rest.
Implement a filter on a pure stream by recursively processing chunks, applying a predicate, returning a stream of filtered elements, and continuing with the rest of the stream.
Learn to build a running sum pipe in fs2 by processing input streams chunk by chunk with scanChunksOpt. Start with zero and emit the updated total after each chunk.
Explore the running max exercise in functional streams for Scala with FS2, building a max over chunks from a neutral element through an accumulator and state updates.
Pull-based streams emit only when asked, delivering elements in chunks for performance, while pools act as stream processes for stateful transformations, and pipes with the through operator build pipelines.
Explore concurrency in functional streams with fs2, learn three concurrent strategies, process elements in parallel with part event map, and save streams using sip, sit right, and pass it.
Explore merging concurrent streams with fs2 and Cats Effects, observe interleaving behavior, error handling on left or right failures, and finite versus infinite streams using merge, drain, and debugging prints.
Explore merging finite and infinite streams in fs2, demonstrating merge options that halt when the left stops, the right stops, or either stops, with concrete run results.
Explore parJoinUnbounded, a generalized concurrency method for combining streams in fs2 by building a stream of streams and pulling non deterministically.
Demonstrate parJoin by using a bounded, two-open-stream strategy on a stream of streams to exhaust finite inputs before starting new ones, and warn about infinite streams.
Create a bounded queue and run multiple producers and consumers in parallel using fs2 streams, join them, and drain the output after five seconds to demonstrate concurrent processing.
Combine two streams concurrently in fs2 to see interleaved outputs; the left stream is main, the right is background, and errors stop the entire stream, unlike merge.
Create a ref initialized to zero, simulate processing 30 items with a processor that updates the ref, and emit progress updates every 100 milliseconds while draining the stream. Run a separate progress tracker stream that reads the current count, prints a progress message with the calculated percentage of items processed, and illustrate why the processor goes on the left and the progress tracker on the right as a background task.
Demonstrate a parallel version of nirvana, processing jobs with parEvalMap to boost concurrency, while comparing unbounded, bounded, and ordered vs unordered execution.
Practice implementing parEvalMapSeq-style extensions on fs2 streams to process jobs concurrently, returning a list of events in io, with bounded concurrency and a second bounded version.
Explore zip in fs2 by combining two streams, handling finite and infinite sequences, errors, and laziness, then observe element pairing and practical results with simple prints.
Explore fs2 zipRight by running a left side-effecting stream and a right stream that prints the local datetime, then returns the right value, illustrating concurrency-ready behavior.
Expose ParZip in FS2 by running two streams in parallel and producing left-right pairs. Compare sequential left-right pulling with parallel racing, noting that emitted tuples stay the same despite concurrency.
Explore fixed-rate streaming in fs2 for scala, using a fixed-rate stream to throttle a processing task and emit values every two seconds, with meter regulating the downstream work.
Explore fixed delay streams in fs2 to enforce a 2-second pause between elements, accounting for processing time. Use the spaced method with duration and stream to compare with fixed rate.
Explore awake every and wait delay streams in FS2, showing how emissions are timed, compare fixed rate and fixed delay semantics, and observe duration impacts on cadence.
Explore concurrency in functional streams, process elements in parallel, and apply time-based patterns using fix, delay, and fixed rate, including combining streams to form meter-like patterns.
Explore communication patterns in functional streams: interruption signals, value transfer between producers and consumers, multi-producer channels with buffering, and publish-subscribe topics and queues with buffering.
Explore signals, a concurrency primitive in fs2 that lets one stream interrupt another. See how a boolean signal, shared between streams, stops a worker via interrupt when, with concurrent execution.
Demonstrates using signals and a ref-based temperature sensor and cooler to communicate alarms through streams, with a threshold and concurrent execution interrupted after 3 seconds.
Explore channels as a concurrency primitive in FS2, showing how bounded and unbounded channels manage multiple producers, buffering, and blocking to prevent deadlocks while streaming data.
Build two sensors with fs2: a shared measurement channel for temperature and humidity, generate random readings, print them, trigger alarms on thresholds, and run concurrent streams with periodic checks.
Explore topics as a concurrent primitive in functional streams, implementing a multiple publisher, multiple subscriber pattern with a topic, buffered subscribers, and interleaved element processing by two consumers.
Explore backpressure with topics by modeling a producer and consumer using a buffer that slows production to prevent data loss, ensuring sequential processing.
Define a car position with coordinates, publish updates to a topic, subscribe with a Google map updater and a driver notifier, and drain the stream after processing.
Explore concurrent patterns in fs2 by implementing a producer and consumer using an unbounded queue, a shared io ref, and stream integration to monitor the ref while running concurrently.
See how backpressure is handled with queues in functional streams for Scala with FS2. A fast producer outpaces a slow consumer, while the queue buffers thousands of elements.
Create a stream from an unbounded queue of options, concatenate with none to signal termination, and read from the queue with a non-terminating read that stops at none.
Design a simple server with a controller that handles post account requests, enqueues data into an unbounded queue, and runs a consumer stream to process and print results.
Explore sending messages between streams via signal channels, topics, and queues, and connect to the outside world by buffering, queuing elements, and processing them with a stream.
Wrap up your journey through fs2, streaming, and functional programming in Scala, reflecting on what you learned and expanding your toolkit.
Many applications involve dealing with large flows of data. Examples are processing files in ETL jobs, reading results from a database or getting a big response from http calls. Handling large amounts of data often means sacrificing either readability or performance.
With streams, you can get the best of both worlds:
- Data is processed using a constant amount of memory, even if the total amount of data is very large
- The processing is built declaratively as if you were dealing with regular Lists or Sequences, with high level methods such as map, filter and flatMap
Furthermore, streams in FS2 are effect-aware. They work in the context of an effect monad like IO, which enables them to do all sorts of useful stuff such as processing elements in parallel, throttling, retrying on failure and many more.
In this course we will turn streams inside out and learn things like:
- Create and combine pure streams
- Add effects to our streams and learn how to compose them
- Handle errors & resources safely
- Apply patterns involving time, such as retries, throttling and debouncing.
- Build our own stream transformations with Pulls and Pipes
- Handle concurrency using many different patterns
- Communicate between streams using primitives such as Signals, Channels, Topics and Queues
Join me in this journey and add yet another amazing tool to your functional programming toolkit!