A new Solidity compiler in Zig

(Note: this article is written without the use of AI)

We have a gift for the Ethereum community: A new Solidity compiler (GitHub)! We don’t believe it’s ready for deploying smart contracts yet, but it can help with your dev workflows, and hopefully can become a testbed for experimentation in the compiler implementation.

The back story

We’ve been building a Solidity protocol at OKcontract Labs: Chainwall. It’s a large codebase with lots of tests (which is supposedly good) but it turns out that compilation or recompilation of the project including those tests took more than 10 minutes! Dozens of times per day, that’s a lot of coffee breaks. This is how the idea of a rewrite of the Solidity compiler came in.

Our compiler, nicknamed oksolc, started from a mechanical port of the original C++ compiler to Zig and therefore is a derived work under the GPLv3. We will cover more details about our approach in the coming months. We used AI assistance, but it’s very very far from “Rewrite Solidity in Zig, make no mistakes”. Our team has experience in building compilers and in formal verification (back when it was the “technology of the future… and staying that way”).

Why Zig?

Optimizing the Solidity compiler has several requirements: Compatibility with the original codebase was imperative, as well the ability to minimize structural changes (ruling out languages such as Rust). We were left a set of 3 candidates: C++, as well as C and Zig.

Zig appeared as the most attractive option, providing nicer modern code without any layer of magic. It provides support for big integers in the standard library, eliminating the need for GMP. We also benefit from direct compatibility with C which we use for example for yyjson, which showed improved performance vs. the Zig standard library. Also, if you read further, there was another major C library that we needed … but no spoilers.

Zig avoids splitting declarations and implementation, and also avoids C++ overloading which makes, in our opinion, the code much more readable and understandable:

bool visit(ContractDefinition const&) override
{
	solAssert(!m_currentConstVariable, "");
	return true;
}

bool visit(VariableDeclaration const& _variable) override
{
	if (_variable.isConstant())
	{
		solAssert(!m_currentConstVariable, "");
		m_currentConstVariable = &_variable;
		m_constVariables.push_back(&_variable);
	}
	return true;
}
// ...

In our zig implementation, we use in a straightforward way:

switch (node.payload) {
  .contract_definition => {
      if (self.current_contract != null) return error.InvalidAst;
      self.current_contract = node;
  },
  .variable_declaration => try self.visitVariableDeclaration(node),
// ...

Memory management is explicit so the code contains many deinit calls, but this enables fine control on the compiler passes allocations:

var children: std.ArrayList(*const AST.Node) = .empty;
defer children.deinit(self.transient_allocator);
try ASTImplementation.appendChildren(self.transient_allocator, &children, node);
// ...

Some structures need to survive the function that creates them, for instance in AsmAnalysisInfo.getOrCreateScope:

pub fn getOrCreateScope(
  self: *AsmAnalysisInfo,
  block: ?*const AST.Block,
) std.mem.Allocator.Error!*Scope {
  if (self.scopes.get(block)) |scope| return scope;
  const scope = try self.allocator.create(Scope);
  errdefer self.allocator.destroy(scope);
  scope.* = Scope.init(self.allocator);
  errdefer scope.deinit();
  try self.scopes.put(block, scope);
  return scope;
}

And then later (in AsmAnalysisInfo.deinit):

var scope_iterator = self.scopes.valueIterator();
while (scope_iterator.next()) |scope_pointer| {
  scope_pointer.*.deinit();
  self.allocator.destroy(scope_pointer.*);
}
self.scopes.deinit();

What changed

First, what should not change: The bytecode output. And beyond that, the compiler JSON output should be byte-for-byte identical.

It is possible that oksolc changes the output in cases we didn’t (fore)see in our tests. If so, please open an Issue and report a bug with a link to a public source repo for reproduction.

Port, cleanup and optimization

As part of the rewrite, we initially sticked to the original compiler architecture, preserving all passes. We then started to optimize datastructures: For instance, we introduced hashed expressions and caching in several places.

We chose to rewrite the CLI from scratch, adding new features like a webserver. We plan to work on many features useful both for developers and auditors.

Also, we removed the legacy compiler paths, only supporting “via-IR”, the modern compiler path using Yul.

We also replaced intermediary Yul output in text form and switch more passes to pure AST transforms. This comes at the expense of subtle changes the pretty-printing of the Yul. There are differences in spaces, temporary variable names, etc. We would love to get feedback from the community on this as we can consider reverting or re-implementing this if it breaks some use cases or tools.

Time for some benchmarks, starting with Uniswap V4:

Parallel compiler

A major feature is the introduction of parallel compilation. Most modern computers have many cores and we should use them. Our parallelism implementation is probably not optimal in its current form and begins after ordered preparation.

Here are the same Uniswap V4 benchmarks:

Incremental compiler

Another major feature is the support for incremental compilation. When working actively on a codebase, recompilation times after smaller changes may matter even more. That’s why we introduced support for incremental compilation.

And this was another reason to choose Zig: We used and embedded SQLite to store efficiently the compiler cache for incremental compilation. In Zig, it’s just a matter of including the SQLite Amalgamation, an integration even easier than using rusqlite.

Here are the results for Uniswap V4 for updating some constants:

All in all, an incremental parallel recompile of Uniswap V4 takes 4.21s (on a Macbook Pro) compared to 59 seconds with the original compiler.

We also tested on Pendle V2 a similar operation with a minimal change:

What’s next?

All in all, the speedups relative to solc for parallel and incremental compilation are:

For Chainwall, the solc time of 709.84 sec is reduced to 138.42s using parallel 4 jobs. Incremental compilation for a small change is 17.2s and a meaningful change impacting 60+ contracts is 99.52s, a 7x improvement. This solved nicely our performance issue while compiling Chainwall.

Beyond that, we believe the oksolc main benefit beyond the current performance improvements is a codebase ready for more experiments in language design and features. We already have lots of ideas about where to take it next.

As a final word, oksolc is clearly not a replacement for the reference compiler, which must be used for deploying contracts onchain. We’re grateful for the extensive amount of work by numerous contributors that went into the Solidity compiler without which none of this would exist.