Briefing

Rust Macro Expansion Performance: Ambiguous Repetitions Cause Cubic Growth

ux
by bal-e.org via polywolf ·

Optimize macro expansions to avoid cubic growth in ambiguous cases.

What to do now

Profile macro expansions and refactor ambiguous patterns to reduce runtime.

Summary

Rust's macro expansion engine uses a breadth‑first search style algorithm to match input tokens against macro arms. When a macro contains ambiguous repetitions, such as multiple $($e:expr),* patterns, the compiler explores all possible parses, leading to cubic runtime growth. The analysis shows that ambiguous grammars can produce dozens of valid parses, and the compiler checks for ambiguity on each invocation. An example with '@' tokens demonstrates exponential memory usage when the number of tokens increases. Despite the ambiguity, the compiler reports an error, but the underlying algorithm still incurs significant overhead. The current implementation is a subset of an Earley parser, which is known to have quadratic or cubic complexity for ambiguous grammars. Optimizing macro expansion could involve caching parsed expressions or refactoring ambiguous patterns. Developers should profile macro expansions and refactor ambiguous repetitions to reduce runtime.

Key changes

  • Macro expansion uses BFS‑like algorithm
  • Ambiguous repetitions cause cubic runtime growth
  • Compiler checks for ambiguity on each invocation
  • Example with '@' tokens shows exponential memory usage
  • Current implementation is subset of Earley parser
  • Ambiguous grammars produce many valid parses
  • Optimizing could involve caching parsed expressions
  • Refactor ambiguous patterns to reduce runtime

Affects

none

Customer impact

Analyzing matches…

Ask about this story

Impact on an agency? Which customers? Compare historically Risks of waiting