A Rust-Based Parser for Accessibility, Correctness, and Performance

pest: A Rust-Based Parser for Accessibility, Correctness, and Performance

Pest is a general-purpose parser written in Rust that prioritizes accessibility, correctness, and performance. It excels at parsing complex languages by leveraging parsing expression grammars (PEG), which offer enhanced expressivity compared to regular expressions.

Getting Started

To start parsing with pest, it is recommended to read the official book. The book provides comprehensive documentation and guides developers through the process of using pest effectively. Additional resources such as the API reference on docs.rs and a fiddle to play with grammars and share them can be found on the official website.

Example

An example of a grammar for a list of alphanumeric identifiers where all identifiers do not start with a digit in pest is as follows:

alpha = { 'a'..'z' | 'A'..'Z' }
digit = { '0'..'9' }

ident = { !digit ~ (alpha | digit)+ }

ident_list = _{ ident ~ (" " ~ ident)* }
// ^
// ident_list rule is silent which means it produces no tokens

One of the strongest features of pest is its meaningful error reporting. Based on the grammar definition, the parser provides automatic error reporting. For example, given the input "123", the parser will generate an error message indicating that an unexpected digit was encountered. Similarly, if the input is "ab *", the parser will report that an identifier was expected. These error messages are obtained from the default Display implementation of the parser result.

Pairs API

The Pairs API allows developers to derive a Parser implementation from the grammar automatically. Parsing with pest returns an iterator of nested token pairs, providing developers with fine-grained control over the parsing process. Each pair represents a combination of the rule that matched and a span of the input. This powerful feature allows for additional processing and manipulation of the parsed data.

use pest_derive::Parser;
use pest::Parser;

[derive(Parser)]

[grammar = "ident.pest"]

struct IdentParser;

fn main() {
let pairs = IdentParser::parse(Rule::ident_list, "a1 b2").unwrap_or_else(|e| panic!("{}", e));

for pair in pairs {
    println!("Rule:    {:?}", pair.as_rule());
    println!("Span:    {:?}", pair.as_span());
    println!("Text:    {}", pair.as_str());

    for inner_pair in pair.into_inner() {
        match inner_pair.as_rule() {
            Rule::alpha => println!("Letter:  {}", inner_pair.as_str()),
            Rule::digit => println!("Digit:   {}", inner_pair.as_str()),
            _ => unreachable!()
        };
    }
}

}

Other features

Apart from its accessibility, correctness, and performance, pest offers additional features such as precedence climbing, input handling, custom errors, and compatibility with stable Rust. These features make pest a robust choice for parsing languages in various domains.

Projects Using pest

pest has gained popularity within the Rust community, and it is being used in various projects and ecosystem tools. Some notable projects include:

• pest_meta (bootstrapped)
• AshPaper
• brain
• cicada
• comrak
• elastic-rs
• graphql-parser
• handlebars-rust
• hexdino
• Huia
• insta
• jql
• json5-rs
• mt940
• Myoxine
• py_literal
• rouler
• RuSh
• rs_pbrt
• stache
• tera
• ui_gen
• ukhasnet-parser
• ZoKrates
• Vector
• AutoCorrect
• yaml-peg
• qubit
• caith
• Melody
• json5-nodes
• prisma

Minimum Supported Rust Version (MSRV)

pest maintains compatibility with Rust 1.61.0 and later versions. This ensures that the library can be used with the latest stable version of Rust, offering developers access to new language features and improvements.

no_std Support

For developers targeting embedded environments or building without the Rust standard library, pest provides support for no_std builds. By disabling the default features of the pest and pest_derive crates, developers can use these libraries in resource-constrained environments.

Special Thanks

Finally, a special thanks to professor Marius Minea for his guidance and to all the contributors who have made pest what it is today. The support and collaboration from the Rust community have been invaluable.

By leveraging the power of Rust, pest offers a reliable, performant, and accessible solution for parsing complex languages. Whether you need to parse domain-specific languages or create your own programming language, consider giving pest a try.

Please feel free to ask any questions or share your thoughts on this powerful parser for the Rust ecosystem!

source