Scala Advanced, Part 3 - Functional Programming, Performance

Delve into more functional programming and the beginnings of advanced practices, techniques, and approaches for part three of the SCANA advanced training.

Explore recursion versus leap's recursion to write more functional code, eliminating explicit loops; examine Scala's Telcel optimization, algebraic data types, trampolines, and Scala tile calls, then recap higher order functions.

Explore why recursion offers a functional, immutable alternative to loops in Scala, contrasting imperative while loops and accumulators with a factorial example and side effects.

Compare imperative and recursive factorial implementations, including self recursion, highlighting termination conditions to prevent infinite recursion. See how deep recursion causes stack overflow, while loops avoid it.

Explore how tail recursion and tail call optimization turn recursive code into an efficient while loop, using an accumulator to avoid stack overflow and boost performance, with optional tailrec annotation.

Explore a fibonacci puzzle that sometimes requires more than one accumulator, revealing a clever trick used in interviews to gauge a candidate's reasoning process.

Examine mutual recursion in Scala through a simple even-odd example where two functions call each other, which tile call detection won't catch, illustrating limitations before deeper context.

Explore a mutual recursion between odd and even, with base cases even of zero is true and odd of zero is false, highlighting slow performance and stack overflow on inputs.

Explore algebraic data types in Scala with a sealed trait bounce, using a trampoline to avoid descending the stack and implement a small domain-specific language.

Explore a functional trampoline in Scala: pattern-match on bounce states, achieve tail recursion without stacking, and implement a generic trampoline that iterates through calls until done.

Explore trampolines and a no-var, pure functional implementation based on an ADT and higher-order functions, enabling endless execution and serving as building blocks for future Scala Util Control Tile Calls.

Explore Scala tail calls and tail recursion optimization with a boolean example, by-name parameters, and the built-in trampoline in tile calls for efficient mutual recursion.

Explore higher-order functions and first-order functions, showing how returning function values powers trampolines and interpreters in functional programming patterns, with hands-on Scala exercises and tests.

Explore advanced functional programming ideas and approaches, and examine patents, in module 13 of the applied Scala course.

Explore options as functors and monads, covering map and flatMap, laws, and implementing a simple option, then learn applicative functors, free monads, and Katz library patterns to apply when appropriate.

Explore how options in a for expression compose an optional person and address to yield an optional zip code, and how map and flat map implement the functor and monad.

Build a simple option container from scratch, with none and item cases and map and flatMap operations that flatten nested options for clean functional workflows.

Apply the functor pattern by implementing isDefined, isEmpty, and map for an optional type. None yields empty with no function call, while some applies the function and returns an optional.

Implement the flat map for an optional by following the type signature and handling empty box, then apply a function from T to optional of U to yield the result.

Explore using optional types in scala with for-comprehensions to extract zip codes from an optional address, compare to option, and discuss product with serializable and type inference.

Reintroduce the guard in a for expression using withFilter to apply a predicate, turning maps and flatMaps into optionals and handling empty results or failures.

Test the guard and the predicate in the worksheet, play with it, and confirm you understand what this predicate is doing here.

explore the monad laws—left identity, right identity, and associativity—in Scala, using flatMap and apply on options, with examples and the role of property-based testing to verify.

Explore applicative functors in Scala, showing how independent option values can be combined with yield and map/flatMap to produce a result or none.

Explore applicative functors with futures and options, show how zip combines independent results to run operations concurrently, and discuss validation use cases with multiple failures.

Examine how Scala libraries like Skarsgård and Cats implement applicative functors, compare cartesian syntax with the deprecated operator approach, and adopt a tuple-based applied map in Cats.

Explore functor and monad patterns in functional programming with the cats library, using for expressions to compose inside a container and capture side effects, failure, state, and logs.

Isolate input output effects in a purely functional way, using cats' io, which stores computations as values until run, capturing side effects and failures safely.

Explore how the io functor and monad enable composing io actions with map and flatMap using a for-expression, yielding a person from name and age, and running with unsafeRunSync.

explore the reader monad as a functional approach to dependency injection, showing how dependencies are accumulated and injected later, with parfit or cupcake options and JVM optimization tradeoffs.

Explore the writer as a journaling tool that logs steps, tracks data provenance, and composes authorization and greeting flows via a monoid.

Learn to model state functionally, avoiding mutable state, using the state functor and monad in cats to thread state through a queue for pure results.

Discover the free monad pattern by lifting database operations (save, search, delete) into an ADT and a queued list, then interpret and execute with swappable backends.

Learn to model a conceptual database with a four-operation ADT (save, load, remove, show state) as a Cats free monad, enabling composable steps and a fold map.

build a composite operation that loads a type t by id, applies a function, and saves the updated item, returning unit via a free monad and handling none with pure.

Compose and manipulate a free end-to-end program that loads, saves, updates, and deletes database entries, demonstrating state changes and composable, functional programming in Scala.

build and run the interpreter that converts a db free to concrete types using foldMap, handling save, load, and remove operations on a simple mutable map and printing state.

Explore running a functional program with an interpreter, evolving state through a database-backed workflow, and applying the principle of least power to balance free types and simple case classes.

Explore how to use or not use macros in Scala. Understand module 14's place in the advanced course and when to apply or avoid macros in Scala.

Explore the first role of macros, the Scala compiler's handling of abstract syntax trees and phases, and define and call macros using the demo egalite and quasi Cote's, with alternatives.

Avoid macros when possible, as the first rule; rely on the Scala type system, the implicit type system, and macro hygiene for safe, disciplined solutions.

Explore abstract syntax trees in Scala by inspecting AST dumps across the type-phase and a graphical dash-y view, showing for expressions, implicit resolution, and macro transformations.

Macros form a post typecheck phase, take the AST and return a modified AST, but cannot fix invalid syntax; use pattern matching or quasi quotes.

Explore a demo macro to convert a first-class scala function into a descriptive string, showing how macros, apply methods, and method handles reveal function behavior.

Set up a named function with a facade that delegates to the underlying function, overriding the two-string representation for readable names, and explore when macros are or aren't necessary.

Generate a no-op macro in Scala, exploring black box macros, the runtime context, and end-to-end expressions. Learn why white box macros are discouraged and how the macro preserves types.

Demonstrate building a macro with generic type parameters that accepts an input type and output type, returning a function and enabling any input-output type combination.

Explore quasi quotes in Scala macros, using scala.reflect.runtime.universe to parse and manipulate abstract syntax trees, pattern match definitions and sequences, and illustrate operator representations.

Explore how quasi quotes convert code to an abstract syntax tree, pattern-match class name, superclass, traits, and body, and reassemble it into new generated expressions.

Describe a macro by using quasi quotes to capture an expression, convert it to a string, and assemble a new named function with the original expression embedded.

Describe narrowing the return type by returning a named function subtype of the original and safely outcasting to simplify type signatures in the code.

Explore code behavior, observe an error placeholder, and decide when to escalate to a compiler error to create a compiler warning.

Explore macro limitations, such as post type checking and pre implicit resolution. Avoid macros when possible, and understand how sugaring and infix forms affect clarity in method positions.

Beware the seductive pull of macros, which can spiral into complexity. Rely on type classes, implicit constraints, and the type system, with minimal macro use as in Shapeless or Magnolia.

Explore the Poza compensators module in Scala Advanced, examine how the polycarbonates library in SCANA fits into core libraries, and compare internal and external dsl for constructing external deals.

Explore building a simple parser using Scala parser combinators: from tokens and regex to abstract syntax trees, with transformers, syntax recap, and Packrat parsing options.

Discover external DSLs in Scala by contrasting them with internal DSLs, covering infix notation, parentheses, and postfix operators. Learn how to build an external language using Polycarbonates.

Develop a basic dialect grammar using a BNF-like specification, defining line numbers, variables, print statements, and simple control constructs like for, next, and go to, with parser-friendly syntax.

Explore simple parser combinators by constructing string token parses and Java-like regex parsers, define keywords such as goto, print, and next, and extend parsers for Java-like languages.

Explore token parsers in Scala advanced part 3, focusing on regex passes with whitespace matching and handling, implicit conversions, and support for decimal numbers, floating point literals, and string literals.

Learn how transformers convert parsed strings into other types, such as integers, by mapping parser output through a function and handling transformation failures.

Learn how to increase the complexity of types using transformers, by building an abstract syntax tree with case classes for line numbers and variables, validated by regex.

Define the statement line algebraic data type using sealed traits, with print, go to, next, and complete line variants, including a line number, print list, values, and optional by value.

Construct and parse a next singleton object, then model go to statements with line numbers using tilde, case classes, and the apply method in Scala parsers.

Explore how BNF translation maps grammar constructs into Scala types, converting colon colon equals to a type T and using pipes for alternatives, repetition, and option handling.

Explore parsing and building a print statement by converting a string sequence into a print list and a print case class, including string literals and optional separators.

Master the for statement in Scala advanced by building it step by step with the compensators library, including optional by part, bipartite structure, and pattern matching.

Assemble a complete line from a line number and its statement into a case class, then parse strings into lines and form a program with success or compiler error messages.

Practice a basic parser with line-based statements, explore compile-time errors and runtime feedback, and see how tokenization, pattern matching, and abstract syntax trees drive the Scala compiler toward bytecode generation.

Explore packrat parsers in Scala, address left recursion and backtracking with the packrat powers trait and lazy values, and note the memory tradeoff while trying module 15 koans.

Explore performance and optimization in Scala, identifying what is and isn’t fast in Scala and when it matters, balancing good style with the need for faster, more efficient code.

Profile first to prove a performance issue, then optimize using Scala's collections API, while loops, tail calls, and primitive usage over boxing. Cache results to avoid repeated computations.

Ensure the code works and is simple to read, then profile and optimize hotspots where most time is spent to achieve cost reduction by speeding the critical paths.

Prioritize well-tested code with regression tests before optimizing, then run benchmarks with 10–20 repeated executions to observe JVM optimizations, not theories.

Explore how scala collections affect performance, with immutable lists slow at appends. Use vectors for fast appends as shown by Fibonacci builds and hot spot optimizations, delivering dramatic speedups.

Compare vector and list performance in functional programming, showing that prepending to the head and avoiding reverse can escape n-squared time, with vectors offering better memory efficiency while remaining immutable.

Optimize fibonacci performance by removing higher-order functions like fold left and other ops, using a primitive long array and while loops, avoiding boxing while balancing safety and readability.

Explore why while loops and arrays are fast in the JVM, including primitive arrays and mutability tradeoffs, and compare fast arithmetic like x times x to library power calls.

Run timing tests to compare register arithmetic with pow calls; X times X is often faster than pow(X, 2), while hot spot optimizations may change this.

Explore bitwise left shifts to speed up multiplying by powers of two and understand how hash code functions can become hotspots, with prime 31 and compiler optimizations accelerating hashing.

Explore loop choices, for expressions, while loops, and tail recursion, plus a bitwise test for even numbers. For runs about 341 ms, while about 58 ms, a 5–7x speedup.

Explore tail recursion versus while loops in Scala, revealing how tail calls are optimized into while on the JVM and comparing performance with map and for each style.

Reduce Scala transcendental function costs by avoiding square roots in distance calculations, using squared distance for sorting and threshold comparisons to boost performance.

Demonstrates optimizing distance checks by using distance squared instead of square roots, achieving 30–40% speedups and emphasizing avoiding sqrt when unnecessary.

Explore profiling to identify hot spots in slow code, then do not optimize until you've demonstrated the problem, fix issues iteratively, and stop when performance is fast enough.

Explore profiling Scala code with the free Oracle profiler called Visual VCM, distributed with OpenJDK; learn two approaches, with sampling usually sufficient for most cases.

Use VisualVM to profile performance by running a minute-long test, enabling CPU sampling, taking a snapshot, and drilling into hotspots to refactor critical code until performance improves.

Profile and optimize a Scala Monte Carlo pi calculation using Visual VM, identify hotspots like Java random, swap in Zol Random, and refactor with fast while loop to boost performance.

Profile and optimize with concurrent futures to speed up pi calculation, moving from single calls to multi-call parallelism. Identify hotspots and CPU usage using iterators, futures, and profiling tools.

Cache results to trade memory for speed, but guard against memory leaks. Use lazy vals and soft values with Guava's CacheBuilder to keep results deterministic in a functional style.