Building a High-level Compiler from scratch

Course overview

In recent compilers implementation, it became popular to translate from one high-level language to another high-level language. Examples might be desugaring new version of JavaScript to an older version of JavaScript, or languages like TypeScript, etc. Such high-level compilers are known today as transpilers, and what is the topic of our course.

It proved to be very practical, since we can fully rely on an existing runtime such as a JavaScript engine, avoid dealing with lower-level constructs like memory, bytecode instructions, etc.

If you would like to explore the world of compilers and implement your own programming language, starting from the high-level compilation is a good hands-on practical approach, and which we apply in this class.

Often related books on compilers go to theoretical aspects, stucking in parsing, regular grammars, etc – not explaining how actually to build a practical transpiler. I believe we should be able to build a transpiler for a full programming language, end-to-end, in 2-4 hours — with a content going straight to the point, showed in live coding sessions as pair-programming and described in a comprehensible way.

In the Building a Transpiler from scratch class we focus on compiling a high-level concurrent programming language with message-passing processes, down to JavaScript. That is, in addition to the deep dive on the transpiling, you will get a better understanding of how concurrent functional languages, such as Erlang, or even an operating system itself with its processes and threads, work today.

Implementing a transpiler will also increase your engineering level, as it touches several aspects of data structures and algorithms.

Prerequisites

An optional prerequisite for this class is the Building an Interpreter from scratch (aka Essentials of Interpretation) course, where we build an AST-interpreter for a full programming language. Unless you already have understanding of how programming languages work at this level, i.e. what eval, a closure, a scope chain, environments, and other constructs are — it is recommended to take the interpreters class as a prerequisite.

Who this class is for?

This class is for any curious engineer, who would like to gain skills of building complex systems (and building a transpiler for a programming language is an advanced engineering task!), and obtain a transferable knowledge for building such systems.

If you are interested specifically in compilers, PL and Type theory, and want to build a transpiler for your programming language, this class is also for you.

What is used for implementation?

We build the transpiler in JavaScript, compiling to JavaScript.

JavaScript, being the most popular programming language, should be accessible for many engineers, and also our goal is to provide a simple and concise implementation, not focusing on specifics of a host language. So the code should be portable to any language of your taste and choice: TypeScript, Rust, OCaml, C++, Python, etc.

Note: we want our students to actually follow, understand and implement every detail of the Transpiler themselves, instead of just copy-pasting from final solution. Even though the full source code for the transpiler is presented in the video lectures, the code repository for the project contains /* Implement here */ assignments, which students have to solve.

What’s specific in this class?

The main features of these lectures are:

• Concise and straight to the point. Each lecture is self-sufficient, concise, and describes information directly related to the topic, not distracting on unrelated materials or talks.

• Animated presentation combined with live-editing notes. This makes understanding of the topics easier, and shows how the object structures are connected. Static slides simply don’t work for a complex content.

• Live coding session end-to-end with assignments. The full source code, starting from scratch, and up to the very end is presented in the video lectures

What is in the course?

The course is divided into four parts, in total of 18 lectures, and many sub-topics in each lecture. Below is the table of contents and curriculum.

Part 1: Transpiler pipeline

In this part we start talking about transpiler pipeline, parsing, code generation, and exploring AST nodes.

Part 2: Functions and Processes

In this part we focus implementing our runtime components, such as Process and Scheduler classes, and discuss cooperative multitasking.

Part 3: Pattern matching

In this part we implement packed data structures, such as Lists and Records, and also build support for pattern matching.

Part 4: Message passing

In the final part we implement the message passing mechanism, do a case study analysis, using the compiled code from Web, and build the final executable.