Common Lisp programming: from novice to effective developer

What we see: the prefix notation. Everything is an expression. The evaluation model: no surprises with functions, but macros don't follow this model. Code is data is code.

How to define variables, at the toplevel or locally. What we see: defparameter, defvar, let, let*, setf, how to lexically re-bind dynamic variables, the gotcha, the alternative (pure functions).

What we see: if, when, unless, cond, case, the or shortcut, the #+(or) "*features*" trick, an example where we load a lispy configuration file.

Common Lisp has more than lists: hash-tables (aka dictionaries), arrays, as well as sets and tree operations. Linked lists are made of "CONS" cells. You should adopt a functional style in your own functions, and avoid the built-ins that mutate data. We see how, and I give you more pointers for modern Common Lisp.

What we see: how to create lists (proper lists, plists and alists). A first warning about the '(1 2 3) notation with a quote.

Lists, continued. What we see: how to access elements: FIRST, REST, LAST, NTH…

What we see: explanation of the different equality functions and why knowing this is necessary when working with strings. EQ, EQL, EQUAL, EQUALP (and STRING= et all) explained. Which is too low-level, which you'll use most often.

What we see: vectors (one-dimensional arrays), multi-dimensional arrays, MAKE-ARRAY, VECTOR-PUSH[-EXTEND], the fill-pointer, adjustable arrays, AREF, VECTOR-POP, COERCE, iteration across arrays (LOOP, MAP).

A "CONS cell" is the building block of Common Lisp's (linked) lists. What do "cons", "car" and "cdr" even mean?

All CL built-in functions accept a :TEST and :KEY argument. They are great. What we see: when and how to use them, when working with strings and with compound objects (lists of lists, lists of structs, etc).

Hash-tables (dictionaries, hash maps etc) are efficient key-value stores. However, as a newcomer, I had them in gripe. They were not easy enough to work with. I show you everything that's needed to work with hash-tables, and my best solution for better ergonomics.

Thinking that '(1 2 3) is the same as (list 1 2 3) is a rookie mistake and can lead to subtle bugs. Demo, explanations and simple rule to follow.

What we see: LOOP in/across and its various accumulating clauses, dolist, for:for, coerce.

What we see: 5 ways in 5 minutes to iterate over a hash-table. (NOTE: you don't imagine all the time that it took me to learn and discover these!!)

What we see: dotimes, the same with loop, looping forever and building our first Lisp REPL.

What we see: the structure of LOOP. Two rules to keep in mind. LOOP smells. We enhance an example from an answer of Advent Of Code 2021.

What we see: DEFUN, returned values, required arguments, optional arguments, key arguments, how to set a default value, how to know if an argument was supplied, &rest, example of apply, example of an inline assertion under a feature flag.

What we see: apropos, documentation #'hello, the difference between 'hello and #'hello (quote, sharpsign-quote), flet and labels.

Multiple return values are NOT like returning a list or a tuple!!! What we see: counter-example in Python, values, multiple-value-bind, nth-value, values-list, m-v-l.

What we see: how to give functions as arguments, member, the :test keyword, map and mapcar, lambda, how to generate functions, setf symbol-function. A word on currying and being a Lisp-2.

What we see: closures (let over lambda and lambda over let over lambda). Notions of lexical scope, functions and their environment.

What we see: how and why setf functions, with our previous counter example and a circle class (we set a new radius given a circumference).

How to write functions that dynamically dispatch on the type of their arguments. What we see (quickly): defmethod, defgeneric (optional).

This exercise comes from Advent Of Code. It is an opportunity to use hash-tables, complex numbers, recursive functions, &key arguments, iteration, equality, development tricks, and more.

We start with a puzzle input, a strings that represents a grid, with obstacles and a guard, patrolling on it. We must parse the string to a data structure, move the guard on a map according to movement rules, and honor our mission.

I give you the code structure, you read code, you try the utility functions in your editor, and write in the missing parts.

What we see: how to load the project thanks to its .asd file, install dependencies, go "in the package", being in a bare bones Lisp REPL or in Emacs and Slime. Appropriate Slime shortcuts. Get a list of the project's dependencies with ASDF programmatically.

What we see: a simple .asd file (flat source tree or a src/ subdirectory), a package definition, two project skeletons.

What we see: a recap on systems. The :cl-user and :cl packages. Creating a package without using :cl symbols. Recovering from symbols name conflicts. Understanding that the reader "interns" symbols from what we type. Printing and counting external symbols of a package. Role of in-package. Exporting symbols.

The Common Lisp Object System is one of if not the most powerful object system available in any language.

In these 2 crash course videos, we see 80% of it. You'll know enough to write your own object-oriented code and read many libraries and projects out there. We will only touch the MOP (Meta-Object Protocol), with which you can create other object systems…

You MUST have a first understanding of object-oriented programming to follow along, even though you can gather quite a lot by watching our code-first classes. Please understand that I don't teach OOP, I explain the Common Lisp way.

MORE VIDEOS TO BE UPLOADED THIS SUMMER (JULY/AUGUST).

---

What we see: DEFCLASS, MAKE-INSTANCE, attributes (aka slots), SLOT-VALUE, slot options (initarg, initform, reader, writer, accessor, documentation), slot-value, generic functions, DEFMETHOD, dispatching on built-in types, how objects are lazily updated, Slime inspector actions, manipulating Slime presentations, unbound slots and SLOT-BOUNDP, Slime shortcuts to create objects…

I chose to re-use the "person" and "child" class examples that we have in the Cookbook. I find it a good analogy for inheritance, and I find it invaluable that you can refer to a written resource with the same analogy. I wrote this Cookbook page, so I'm glad to put it to good use again ;) However, in this video we discover the travel in space that we don't see in the Cookbook ;)

What we see: inheritance, multimethods, :around, :before and :after methods (think signals and overwriting default methods in other languages, that allow to control what happens when a method is called, if it is called at all), their order of execution, a Slime shortcut to export all symbols of a class at once…

Change the default printed representation of objects.

What we see. PRINT-OBJECT, with print-unreadable-object, the object type, the object identity, classic gotchas.

We give another pass, slower, to DEFCLASS, slot options, MAKE-INSTANCE, and to the fact that accessors are generic functions.

You can skip this one if the crash course was crystal clear.

What we see: writing our own MAKE-PERSON terse constructor. Adding some logic before the object creation, doing side-effects after the object creation: towards INITIALIZE-INSTANCE.

What we see: defining a :before and an :after method of INITIALIZE-INSTANCE for our person class, in order to do the same logic than with our custom constructor, but with a built-in CL Object System mechanism. Note that using INITIALIZE-INSTANCE isn't a must, only a "can", that you can use for your own classes, or to control the creation of objects from other systems.

What we see: how to inherit from multiple parent classes and who takes precedence, when the parents define the same slot with each a default value. Quick illustration. We use what is known as a mixin class to add functionality to our class.

What we see: the use of DEFGENERIC and DEFMETHOD, either separately, either together. DEFGENERIC has a couple advantages in regards to documentation and keeping your code in sync with your image.

MORE VIDEOS TO BE UPLOADED THIS SUMMER (JULY/AUGUST).

What we see: the default :allocation :instance VS :allocation :class. How to automatically count how many objects of a class are created.

What we see: the ERROR and WARNING functions, MAKE-CONDITION and SIGNAL. What are simple conditions good for?

What we see: HANDLER-CASE, how to use it with built-in conditions types (division-by-zero…) and with conditions from third-party libraries. The importance of the class precedence list. How to inspect our Lisp image to find available error types.

What we see: DEFINE-CONDITION, :report, :initargs, :initform, :reader and :accessor. Example of a locked package. How to print our error message to the end user.

A quick demo of a very useful macro, coupled with a pattern I use a lot. How to build this macro with HANDLER-CASE.

What we see: what is a call stack, how HANDLER-CASE unwinds it, what HANDLER-BIND does better, its syntax, a practical example. A word about restarts and the INVOKE-DEBUGGER trick for development.

NOTE: I AM IN THE PROCESS OF PUBLISHING THE REMAINING VIDEOS ON MACROS.

Macros do not evaluate their arguments and expand to new code at compile time. What does that mean? A quick intro before diving deeper.

Lisp macros are NOT manipulating text, unlike C. Text leads to many unnecessary problems. We have a fun tour of a trivial need yet complicated issue in C that is easily done in Common Lisp.

QUOTE does not evaluate its argument.

What we see: how to use QUOTE outside macros. Data takes the shape of code. We pair it with eval and we go full circle. We introduce the need to extrapolate values inside a quote.

What we see: how we extrapolate variable values. How they can help create data structures. Real world examples.

Four tips to recognize if you are using a function or a macro, and why it matters.

Macros do NOT replace functions!

What we see: they are not higher-level functions. The subtle but logic need to re-compile functions using macros.

Introducing MACROEXPAND.

Keeping compile-time computing in mind (more on that later). A look at a function's disassembly. So… you might not need a macro yet ;)

What we see: when use it, understanding the common error messages, passing body forms to our macro. Our first macro model.

What we see: how &body differs to &rest. Macro parameters: lots of possibilities, but some conventions carry meaning. Our own DOLIST macro. Our second macro model you can follow.

We build our first macro with backquote and comma-splice, even a quote followed by a comma. We use macroexpand.

What we see: what is variable capture and how to avoid it. Writing our own REPEAT macro. A little discussion about Common Lisp VS Scheme macros. GENSYM can be used outside macros too.

At this point you know enough to write all common macros. See the exercises for easy and not-so-easy ones.

We saw there can be subtle pitfalls when we write a macro. This pattern allows to offload most of the work to a function, which presents many advantages. We demo with our REPEAT macro.

When writing macros, we have the full power of Common Lisp at compile time. This gives great tools to the developer: early type errors and warnings, faster runtime.

What we see: a simple example, writing a scientific macro for conversion of unit at compile time, existing libraries for that, introduction to dispatching macro characters and reader macros.

What we see: other languages don't have macros but can manipulate Abstract Syntax Trees. Code as lists of symbols is not the same, we would need a third-party library to manipulate a Lisp AST proper. This doesn't prevent to develop crazy macros though, see this library adding Haskell-like type checking on top of Common Lisp, in pure CL macros.

defstar allows to specify a function's arguments' types, Serapeum's ecase-of does exhaustiveness type checking. At compile time, of course.

A symbol macro is not your everyday Lisp development tool, but it expands your toolbet. Again.

Macros occur at compile-time. But Common Lisp blurs the lines between read time, compile time and run time. This allows to execute code at READ time.

Macrostep is an editor extension that helps understand our macro expansions. It is only available in Sly and Lem. We demo with the Lem editor.