Python 3: Deep Dive (Part 2 - Iterators, Generators)

Explore Python sequences in depth, covering indexing from zero, slicing rules, slice objects, mutability, copying—shallow and deep—custom sequence types, sorting with sort keys, and list comprehensions.

Explore Python sequence types, including mutable lists and immutable strings, tuples, range, and bytes; learn zero-based indexing, slicing, and the iterable vs sequence distinction, with concatenation and repetition caveats.

Explore Python sequence types, including lists, tuples, and strings, and learn indexing, iteration, mutability differences, and common operations like concatenation, repetition, join, enumerate, and slicing.

Explore mutating mutable sequence types in Python, including in-place edits with append, extend, slice assignments, and the differences between concatenation and mutation.

Explore mutable sequence types like lists, mutate and extend them, understand id and object mutability, and differentiate between in-place updates, aliasing, and copying techniques.

Compare lists and tuples, examining immutability, constant folding, and how Python optimizes tuple creation. Explore memory efficiency, copying behavior, and element access performance in these sequence types.

Explain why Python uses zero-based indexing and how slice bounds work as inclusive lower and exclusive upper bounds. Show how to describe integer ranges and split sequences with slicing.

Explore how to copy sequences and objects safely, distinguishing shallow and deep copies, understanding mutability, and using Python's copy module to avoid unintended mutations.

Demonstrate copying sequences with shallow copies, list comprehension, slicing, and the copy method, comparing memory addresses. Explain deep copies using the copy module to detach nested objects.

Explore advanced slicing in python, including start, stop, step, and negative indices, and learn how slice objects and mutable versus immutable sequences govern extraction, assignment, and out-of-bounds behavior.

Explore Python slicing with slice objects, start, stop, and step; learn extended slicing, none usage, negative steps, and computing equivalent ranges via slice indices for debugging.

Build immutable custom sequence types by implementing __len__ and __getitem__ to return elements by index, handle negative indices and slicing, and raise index errors; enable Python iteration and bracket access.

Create custom Python sequences by implementing dunder get item and dunder len, support iteration and slicing with index handling; explore a fibonacci sequence with memoization.

Explore in-place concatenation and in-place repetition in Python, comparing plus versus plus-equals on lists, tuples, and strings, and learn how mutation affects memory addresses and object creation.

Understand in-place concatenation and repetition in Python, compare mutable lists and immutable tuples, and see how '+=' mutates or rebinds objects across iterables such as ranges and sets.

Mutate mutable sequences with slicing, including regular and extended slicing, via in-place assignment, deletion, and insertion; learn length rules and how iterables like lists, tuples, and sets replace slices.

Explore slicing assignments on mutable sequences in Python, including replacement, deletion, and insertion via empty slices, while noting that immutable sequences cannot be mutated.

Explore how to implement mutable custom sequence types by overloading plus, in-place add, and repetition; handle item assignment, slicing, contains, and deletion, plus append, extend, and pop.

Explore overloading plus, plus equals, and multiplication operators in custom sequences to implement concatenation, in-place updates, and duck-typed compatibility, including the contains operator.

Explore building custom point and polygon sequences in Python, enforcing real coordinates, enabling unpacking, and supporting len, get item, addition, in‑place addition, append, extend, and insert.

Sort sequences with Python's sorted, using sort keys and the key parameter. Sort in ascending or descending with reverse, and recognize stable sort and timsort when keys determine order.

Explore Python's sorted function and list.sort for sorting sequences, using key functions, with natural ordering, and handling dictionaries, complex numbers, and custom objects with stable ordering.

Explore list comprehensions in Python, transforming and filtering iterables with transformation, iteration, and optional conditions. Learn about scope, closures, and nested comprehensions, and how to keep code readable.

Explore list comprehensions in Python, replacing append loops with one-liners and filters. Learn nested and multi-line comprehensions, closures, and practical examples like squares, multiplication tables, zipping, and dot products.

Build a polygon class for regular, strictly convex polygons with apothem, circumradius, edge length, interior angle, area, and perimeter; plus a polygons sequence supporting highest area-to-perimeter ratio and vertex-based ordering.

Design and test an immutable polygon class in Python, with n and r, computing side length, interior angle, apothem, area, and perimeter, plus comprehensive unit tests using asserts and isclose.

Implement a polygons sequence type from three to a maximum edge count, providing length, representation, indexing, and a max efficiency polygon based on area to perimeter ratio.

Explore iterables and iterators, define the iterator protocol, differentiate sequences from sets, and learn to consume iterables and create your own iterators with lazy evaluation.

Learn to iterate collections in Python by using the next item concept, stop iteration for finite data, and build custom iterables with __next__ for use in for loops.

Learn to iterate collections without indexing by implementing a next method and stop iteration, then use length and __len__ to manage exhaustion and restart with a new instance.

Discover how Python iterators work by implementing the iterator protocol with __iter__ and __next__, raising StopIteration to finish, enabling for loops and comprehensions, and noting exhausted iterators cannot restart.

Implement the Python iterator protocol with __iter__ and __next__, manage exhaustion with stop iteration, and explore using iterables via for loops, enumerate, and sorted.

Learn how iterables and iterators differ, implement the iterable protocol with __iter__ returning a new iterator, and separate data from iteration to avoid exhaustion.

Learn to implement iterables and iterators in Python by separating data from the iterator, implementing the iterator and iterable protocols, and using next and stop iteration.

Demonstrate manual consumption of iterators using iter and next, then parse cars.csv with headers and types, cast values, and build named tuples with zip.

Explore cyclic iterators and infinite iteration by building a custom cyclic iterator that wraps a finite sequence with modulo, demonstrates zip usage, and contrasts finite versus infinite loops.

Learn lazy evaluation and lazy iterables in Python, cache deferred properties, and generate values on demand, from factorial iterables to infinite sequences, lazy loading of posts, and generators.

Explore lazy evaluation in iterables through a circle class with a lazily computed, cached area and a factorials iterator, illustrating finite and infinite lazy sequences and upcoming generators.

Explore Python's built-in iterables and iterators, and distinguish between an iterable and an iterator. Examine lazy evaluation in range, zip, enumerate, open, and dictionary methods keys, values, items.

Explore built-in functions that return iterables and iterators, such as range and zip. Understand lazy evaluation, how open creates an iterator for files, and how enumerate and dictionary keys differ.

Explore sorting iterables in Python by using sorted on any iterable, and build a finite random ints iterable with a seed for reproducible sequences, then sort in reverse order.

Learn how Python's built-in iter function creates iterators from iterables and sequences. Explore how sequence types use getitem or the iterator protocol to iterate.

Explore the iter() function and how it yields iterators from iterables, including sequence types with __getitem__ instead of __iter__. Learn to build custom iterators and test iterability with try-except.

Learn to build iterators from any callable using closures and sentinel values, use Python's iter to stop at sentinel, and explore examples with counters, random numbers, and a countdown.

Learn how delegating iterators expose iteration by deferring to an underlying iterable. The example creates a cleaned list of person names and iterates via iter on that list.

Explore reversed iteration in Python for sequence types and iterables using slices, range, and the built-in reversed function. Implement dunder reversed and lazy iterators, illustrated with a deck of cards.

Implement a lazy card deck as an iterable with a dedicated iterator, then apply reversed iteration on both iterables and sequence types, including a custom __reversed__ and a reverse flag.

Learn how iterators behave as function arguments, how they get exhausted, and how to safely reuse data by distinguishing iterables from iterators, using with statements, and defensive checks.

Refactor the polygon and its sequence into lazy, cached computations and a lazy iterator, preserving the external interface while converting the sequence to an iterable with on‑demand polygon creation.

Make polygon properties lazy and cached to avoid repeated calculations of interior angle, side length, apothem, area, and perimeter, validated by unit tests showing goal one accomplished.

Convert the polygons sequence into a lazy iterable by implementing a polygon iterator that yields on demand using M and R, with proper iterator protocol and stop-iteration handling.

Explore generators as iterators and learn how generator functions act as generator factories returning generators. Compare generator expressions and list comprehensions for concise syntax and understand performance considerations across generators.

Explore yielding and generators in Python, turning ideas into lazy iterators that emit values with yield, resume with next, and handle stop iteration.

Explore the yield keyword and generator functions in Python by building iterators, including factorial examples, closures, and iteration protocols to understand exhaustible iterators and creating simple generators like squares.

Explore Fibonacci with recursive and non recursive approaches, compare memoization using LRU cache, and implement iterators and generators for efficient, non recursive Fibonacci generation.

Create an iterable from a generator by implementing the iterable protocol and returning a new generator each time; generators exhaust, and enumerate demonstrates lazy evaluation.

Create iterables from generators to restart iterations without reinitializing them; implement a squares class whose __iter__ returns a new generator, and note that enumerate can exhaust an iterator.

Explore building a Python card deck using generator functions and yield for lazy evaluation, transforming it into an iterable with an iterator and supporting reverse iteration.

Explore generator expressions and list comprehensions, including nesting and if statements, and compare lazy evaluation with eager evaluation to understand memory and performance trade-offs.

Explore generator expressions and list comprehensions, contrast their lazy evaluation with eager lists, and compare performance and memory usage through examples like multiplication tables and Pascal's triangle.

Explore yield from as a way to delegate yielding to other iterators and chain multiple data sources, demonstrated with reading lines from three CSV files.

Explore yield from with nested generators and generator expressions to build and traverse matrices, then apply generators to multi-file data, demonstrating separation of concerns and yield from simplifications.

Create a lazy generator to read the NYC parking tickets CSV, derive headers from the first row, strip newlines, and yield typed named tuples while counting violations by make.

Learn to parse a CSV with Python using iterators and generators, build a named tuple from headers, and yield validated rows with robust type conversion.

Compute car make violations by streaming data with a lazy generator, counting with a default dict, and returning a sorted dictionary of makes by descending counts.

Explore Python iterators and generators with built-in functions and itertools, covering slicing, filtering, chaining, mapping, reducing, infinite iterators, zipping, and combinations.

Explore aggregators that compute a single value from an iterable, such as min, max, and sum, and learn how any and all use predicates and truth values.

Demonstrate Python aggregators min, max, and sum through a squares generator, and examine how any and all handle truthiness and iterator exhaustion in real data.

Explore slicing general iterables with ice slice, a lazy iterator that yields required elements using start, stop, and step.

Explore Python slicing for sequences and iterables, using square brackets, a slice object, and a start-stop slicer with a factorial generator.

Explore selecting and filtering iterables in Python using filter and generator expressions, predicate logic and truthiness, including takewhile, dropwhile, and compress as lazy iterators.

Explore how to build and filter generators in Python, using predicates to select odd cubes, apply filter and filter false, and leverage takewhile, dropwhile, and compress for lazy data selection.

Explore infinite, lazy iterators from itertools, including count, cycle, and repeat, and learn how to constrain or repeat values while noting memory behavior and use cases.

Explore infinite iterators with itertools' count, cycle, and repeat, learn slicing techniques, handle diverse numeric types, and use cycle to deal cards while avoiding shared list references.

Explore chaining and teeing of iterables with itertools' chain and tee, demonstrating lazy concatenation, unpacking pitfalls, and using chain.from_iterable for nested iterables.

Explore chaining of multiple iterables and iterators with Python's itertools.chain, generator expressions, plus custom chain implementations, unpacking, and the lazy behavior of tee for copies.

Explore mapping and accumulation in Python by applying callables to iterables with map, and reducing to single values using sum, min, max, reduce, and accumulate.

Explore map and star map for applying functions to iterables, unpack tuples, and compute squares; then use reduce and accumulate for running results with optional start values.

Explore how the zip function zips multiple iterables, yielding tuples and stopping at the shortest, while zip longest uses a fill value (none or -1) to cover gaps.

Learn how zipping in Python creates a lazy iterator over inputs, yields tuples from the shortest iterable, and use zip longest with fill values alongside generators and infinite iterables.

Group data by make using the EDA tools group by, count models per make with a default dict, while handling csv data, slicing, and lazy iteration.

Explore combinatorics functions in the EDA Tools module, including permutations, combinations, and cartesian products, implemented as lazy iterators and generators.

Explore Cartesian products and itertools techniques to generate grids, matrices, and multi-dimensional axes using product, tee, takewhile, and counting methods, with practical dice and grid examples.

Explore permutations and combinations with Python's itertools, distinguishing with and without replacement, and demonstrate brute-force methods and Cartesian products to count outcomes and compute odds.

Create lazy iterators for four csv files using csv.reader, align by SSN into a namedtuple, filter stale records before 2017-03-01, and compute gender-based vehicle make counts.

Learn to build a lazy csv data pipeline in Python: define constants, create a csv parser, and generate named tuples by applying per-field parsers to each row.

Combine data from four files into a single record by using a parallel iterator, compress fields to drop repeated Social Security numbers, and construct a named tuple with merged fields.

Achieve goal three by adding a filtered inner combine to iterate and filter the four-file data by last updated date using a cutoff date.

Group data by vehicle make and gender, using sorting and itertools.groupby to count groups, address generator exhaustion, and refactor into reusable functions.

Context managers in Python define enter and exit phases to manage resources, enabling safe file handling through the with statement.

Learn how context managers in Python automate setup and cleanup with the with statement, implement enter and exit, and handle exceptions to either silence or propagate errors.

Learn how finally and context managers work in Python, including enter and exit methods, the with statement, and safe file handling with open and custom context managers.

Learn the caveat of lazy iterators with context-managed files: returning a csv reader closes the file; use yield from to preserve laziness or materialize with a list, with memory trade-offs.

Explore how a class can implement both the context manager and iterator protocols, using open and with statements to manage a file, while also supporting iteration and proper closing.

Explore advanced context manager patterns beyond open close, including start stop, transactions, and timing, plus standard out redirection; also explore tag and list maker context managers with nesting.

Explore practical uses of context managers beyond file handling, including decimal context for precision, temporary context changes via enter/exit, timing blocks, redirecting stdout, html tag insertion, and re-entrant patterns.

Explore how generator functions mimic context managers by yielding a resource and performing cleanup on exit, using the with statement to work with a file.

Create a generator-based context manager by yielding inside a try and cleaning up in a finally, then exit with next calls and wrap pattern in a class for with statements.

Discover how to create context managers from generator functions in Python, using a decorator and the contextlib approach to yield resources and ensure cleanup.

Learn to turn a generator into a context manager with a decorator, including enter, yield, and finally cleanup, plus examples like file handling and redirecting standard out.

Explore nested context managers for handling multiple files with with statements or custom context managers, and learn to manage dynamic resource lifecycles using contextlib's exit stack.