An introduction to metaprogramming with Elixir

The Elixir programming language is incredibly versatile. Built on the Erlang VM, Elixir was designed with high-concurrency and low-latency in mind. It’s a great choice for developing large-scale web sites and web applications, as well as embedded systems and distributed applications. And whether you know it or not, Elixir easily facilitates the advanced technique of metaprogramming. By learning metaprogramming, we can extend Elixir’s functionality even further to suit our project’s needs.

Today, we’ll introduce the topic of metaprogramming in Elixir. We’ll take a look under the hood to learn how the abstract syntax tree and macros work together to enable metaprogramming in Elixir.

We’ll cover:

• What is metaprogramming?
• Metaprogramming in Elixir
  • Abstract syntax trees
  • Elixir macros
• Wrapping up and next steps

What is metaprogramming?#

Metaprogramming is an advanced programming technique through which we can write programs that treat other programs as their data. Metaprogramming allows programs to read, generate, and modify other programs. Furthermore, these programs can modify themselves in significant ways.

With metaprogramming, we can:

• Shift computations from run-time to compile-time
• Enable code generation with compile-time computations
• Write code that modifies itself

Metaprogramming gained popularity through list processing languages such as LISP in the 1970’s and 1980’s. Elixir is one of many languages that offer metaprogramming today.

Metaprogramming in Elixir#

There are two components that work together to facilitate metaprogramming in Elixir:

• The abstract syntax tree (AST)
• Macros

Let’s discuss how these components help extend the functionality of the Elixir programming language.

Abstract syntax trees#

The AST data structure works under the hood to support the compiler in most languages. During the compiler’s syntax analysis phase, source code is represented through this structure as an intermediate representation that sets the foundation for code generation.

Metaprogramming in Elixir relies on the ability to manipulate and inspect ASTs. We have the unique opportunity to access the AST when working with Elixir code. Being able to access these program internals lets us operate closer to the level of the compiler than ever before. In the Elixir programming language, these intermediate representations of the AST are also referred to as quoted expressions.

Every Elixir expression is represented by a node in the AST. With the exception of basic values, each node or expression is represented by a tuple consisting of these three elements:

• The first element: An atom indicating function call
• The second element: Metadata
• The third element: Function arguments

For instance, the function call sum(2, 4, 6) is represented as the following tuple or quoted expression: {:sum, [], [2, 4, 6]}

Elixir macros#

Macros allow us to extend the functionality of the Elixir programming language by injecting code into the application. They are special functions that return a quoted expression, which we can insert into our application code. When the compiler finds a macro, it recursively expands the macro until no macro calls remain in the code. We’ll discuss two macros as an introduction to metaprogramming in Elixir.

quote#

quote is instrumental to code generation in Elixir. We can use the quote macro in the iex shell to output the quoted expression for any Elixir expression. For instance, when applied to the code example from the previous section:

quote do: sum(2, 4, 6)
⇒ {:sum, [], [2, 4, 6]}

unquote#

While the quote macro allows us to access the AST, the unquote macro allows us to modify the AST by injecting new code or values into it. unquote evaluates the expression it’s given and injects the resulting AST into a quoted expression. For instance, in this code example, we unquote num and inject it into the quoted expression (or AST) of the function call sum(10, num).

num = 20

quote do: sum(10, num)

quote do: sum(10, unquote(num))

⇒ {:sum, [], [10, {:num, [], Elixir}]}
⇒ {:sum, [], [10, 20]}

A note on using macros#

Macros are considered the building blocks of Elixir. We can use macros to write effective and readable code. They’re great for domain-specific languages (DSLs) like Phoenix and Ecto. They help us carry out complex tasks and build custom language features.

You’ll often hear that one of the first things you should know about metaprogramming is: Don’t use macros. As with most advanced programming techniques, the power that Elixir macros give us can lead to errors if handled incorrectly. Metaprogramming mistakes can result in program errors that are difficult to debug. While the disclaimers are not unfounded, they shouldn’t deter you from learning the powerful technique of metaprogramming. With the right dose of caution, you can leverage Elixir’s macro system to write less code and develop highly performant applications.

Wrapping up and next steps#

As an Elixir programmer, you have the opportunity to extend Elixir’s functionality through metaprogramming. You can leverage Elixir macros to create new language features and DSLs, while eliminating the need for recompilation when your code encounters new situations.

To help you get started with metaprogramming, we’ve created the Metaprogramming in Elixir course. This course provides a safe environment where you can get hands-on metaprogramming experience in Elixir. In addition to covering the basics we’ve touched on today, you’ll learn how to perform advanced code generation, generate functions from remote APIs, create an HTML DSL, and write a test framework.

Continue reading about Elixir#

• A beginner’s guide to the Elixir programming language