May 16

C++Now 2015

C++ Now 2015. Today is the official release of Spirit X3, aka Spirit 3.0.0. X3 will be in beta, coexisting side by side with Qi, Karma, Lex and Classic.

If you are a Qi user, the best place to start are the examples, especially rexpr and x3 fun.

Remember to use a compiler with c++14 capabilities:

  • clang 3.4 or greater
  • gcc 4.9 or greater


Docs: X3 Documentation

Slides: Using X3 Slides

Spirit X3 Source


Use the develop branch.

Zip of develop:

You will find boilerplate (getting started) projects in: example/x3/rexpr
There are two directories.

  • rexpr_min – contains a single file to start things off.
  • rexpr_full – contains a structure similar to the one in the presentation complete with tests.

X3 Fun



Sep 03

Spirit.Qi numeric parsers are the fastest in the world of C++ and surpasses even hand-written low level parsers such as atoi and atof written in C. This is old news actually. The numeric parsers are highly optimised from the very start, when I started writing Spirit V2. The tests were written in 2009, yet are still very relevant now. Here are the benchmark results from the same tests I ran today on a Mac using Clang. The benchmark code is in Github (qi/workbench):

Integer tests

atoi_test:  4.7452590000 [s] {checksum: 213ce33a}
strtol_test: 4.5116520000 [s] {checksum: 213ce33a}
spirit_int_test: 1.4126110000 [s] {checksum: 213ce33a}

Double Tests

atof_test: 3.2435210000 [s] {checksum: 84a4f7d}
strtod_test: 3.1327660000 [s] {checksum: 84a4f7d}
spirit_double_test: 0.7237810000 [s] {checksum: 84a4f7d}

Mar 01

Cool, so it seems Spirit is also useful for blind musicians.

BMC — Braille Music Compiler

BMC translates between braille music code and visual music notation. It uses Boost.Spirit to parse braille music code into an AST which is later enhanced with various algorithms to resolve ambiguities inherent to braille music code. It can currently convert to LilyPond source format, which can later be translated to a PDF and MIDI representation by LilyPond. BMC aims to become an universal translator between tactile and visual music notation, eventually covering both directions. We are looking for developers with a background in music notation who might be interested to contribute to this unique free software project which aims to bridge the gap between blind and sighted musicians.

Dec 13

Boost C++… After more than a decade, I finally retired Phoenix-2 from the Boost Spirit code base. I feel sad. It’s like farewell to a good friend. Onwards to Phoenix-3.

Sep 30

Here’s something cool: SpiritKaleidoscope is a rewrite of the LLVM-Kaleidoscope tutorial using boost::spirit parser. The author is targeting on making it a complete step-by-step tutorial … still work in progress, though. Check it out: SpiritKaleidoscope

Aug 04

A customization point is a code construct that the user can leverage to specialize how a particular library action is handled. A common way of implementing this in C++ is to define a template with the default behavior, and let users specialize it for their own types —e.g., std::hash—. This is the story of Spirit X3 and how it lets you specialize customization points without ever leaving your own namespace, sort of…

I Will Always Find You – Tales of C++

Feb 23

Oh, in case anyone wants to follow the development of X3, here are the Github links:

The full attribute mechanism works + basic C++lambda support. I’ve ported one example (calc4.cpp) which parses to an AST. The AST remains the same. Only the way the grammars are written had to change.

For a teaser: here’s GCC times:

SpiritX3: TOTAL :   4.27 secs
Spirit2:  TOTAL :  10.00 secs

Even faster at CT than the first Spirit3 attempt. Runtime speed? I expect it to be faster than Spirit-2. The rules have changed (pun intentional). Now, there’s no longer the need for virtual functions and auto is used extensively. I expect code size to be smaller too because the compiler can generate more efficient code.

Here’s the calculator grammar (based on calc3.cpp):

//  The calculator grammar
namespace calculator_grammar
    using x3::uint_;
    using x3::char_;

    x3::rule<class expression, ast::program> const expression;
    x3::rule<class term, ast::program> const term;
    x3::rule<class factor, ast::operand> const factor;

    auto const expression_def =
        >> *(   (char_('+') >> term)
            |   (char_('-') >> term)

    auto const term_def =
        >> *(   (char_('*') >> factor)
            |   (char_('/') >> factor)

    auto const factor_def =
        |   '(' >> expression >> ')'
        |   (char_('-') >> factor)
        |   (char_('+') >> factor)

    auto const calculator = x3::grammar(
            expression = expression_def,
            term = term_def,
            factor = factor_def

using calculator_grammar::calculator;
Feb 23

My proposal “Inside Spirit X3. Redesigning Boost.Spirit for C++11” has been accepted for presentation at the C++ Now! 2013 in Aspen Colorado. So after a 2 year hiatus, I’m back to Aspen.

Here’s the abstract:

Inside Spirit X3.Redesigning Boost.Spirit for C++11
Joel de Guzman, 2013

The hugely successful BoostCon ’07 ’08, ’09 and ’10 Spirit talks provided walk-through presentations and tutorials on how to use Spirit. This, time, I propose a presentation that will focus on the design and implementation of Spirit. But to add more to the thrill, I will present a major redesign of Spirit from the ground up, taking advantage of the new C++11 features. One important goal of this experimental version of Spirit (named X3) is to bring back the elegant simplicity of “Classic” Spirit, which was somehow lost with the complexity of Spirit-2 primarily due to the lack of important language features that are just starting to appear in C++ compilers. In this 90-minute presentation, I would like to get down and dirty with Modern C++11 code, and along the way, share my experience as well as expose some of C++11’s shortcomings and my wishes for C++1y.

Hope to see you there!

Aug 10

We’ve had a Spirit IRC channel for a few months now. It started out as a semi-private channel. Well, we made it officially public now. You can get Spirit support via the ##spirit or #boost IRC channels at freenode, where you will often find knowledgeable people willing to help with your Spirit questions. The #boost channel is for anything Boost related, including Spirit. The ##spirit channel is solely for Spirit and Spirit related libraries (Fusion, Phoenix and Wave).

See you there!

Jul 24

Spirit is 10 years old!

It’s hard to pinpoint exactly the birthday of Spirit. Looking back, Spirit 1.0 which was uploaded to SourceForge in July 27, 2001 contains this comment in its main header file:

8/28/1999    Original Implementation [ run time polymorphic version ] (JDG)
4/30/2001    Template meta-programming implementation (JDG)
5/13/2001    Major redesign using iterators (JDG)
5/26/2001    Port to G++3.0 and BCC 5.5.1 thanks to Vladimir Prus
5/27/2001    Bug fixes in Difference and Xor classes (JDG)
5/30/2001    Added Iterators (JDG)

Continue reading »

Jul 23

Mike Lewis posted a marvelous experience report dubbed ‘Optimizing Boost Spirit – Blazing fast AST generation using boost::spirit’. He describes how he took an old compiler for the Epoch programming language (which was based on Spirit.Classic) and tuned it for performance using Spirit.Qi and Spirit.Lex. His results are exceptional, he got roughly a thousand fold speedup compared to the old version. The complete code for his compiler can be downloaded from here.

Continue reading »

Tagged with:
Jul 12

Spirit V2.5 is now available as part of the recently released Boost V1.47. I suggest you look at the What’s New documentation page for a list of things changed. This is a very important release for Spirit, mostly in the area of feature consolidation and less so by adding new functionality. It brings a lot of unification and quite some speedup in the area of attribute handling, both for parsers (Qi) and generators (Karma). Most of the newly added features are in the area of unification of the overall user experience as well. The things added to the Lexer should finally resolve some long standing requests. Most importantly, we are very excited about having added full compatibility with the newly released Phoenix V3 library.

Generally, all changes are supposed to be fully backwards compatible. If you run into problems with your existing code, please tell us by leaving a comment or by sending a message to the Spirit mailing list (as described on our Support page).

Tagged with:
Jun 12

Here’s another cool Spirit talk. This time from Bryce Alexander Lelbach:

utree, a recent addition to the Boost.Spirit codebase, is a generic data structure designed to represent abstract syntax trees. Bindings to Qi and Karma make utree a powerful tool for parser and generator development with Boost.Spirit. This presentation would demonstrate the usage of utree and Spirit to build and manipulate an abstract syntax tree for four parsing/generating use cases: XML, symbolic expressions, JSON, and C-like source code. The details of utree’s integration with Spirit and their implications for writing utree-centric Spirit parsers and generators would be discussed. Additionally, design patterns for compiling a utree AST to other internal representations (DOM tree for XML, function tree for a Scheme-like language for symbolic expressions, associative arrays for JSON objects, a simple VM bytecode for mini-C source code) would be covered.

Here are the slides:


Jun 08

Here’s another BoostCon video uploaded by Marshall Clow. This one is about Phoenix V3, by Hartmut Kaiser:

The slides for this talk can be found here:

Phoenix will be the next generation of creating unnamed, inlined polymorphic function objects. With V3 we combine the functionality of Boost.Bind and Boost.Lambda, and arranges it into a new library. By writing this new library, we were able to fix some limitations of the aforementioned libraries without breaking backwardscompatibility. The purpose of the talk will be to outline the importance and elegance of functional programming (FP) in C++. The first part of the talk will give an introduction into the Domain Specific Embedded Language (DSEL) we defined with Phoenix. A DSEL is built with the help of regular C++ function and operator overloads. For Phoenix we defined such a language that emulates C++, to give potential users a low entry into the world of FP. While a lot of existing C++ code relies on higher order functions (better known as function objects), e.g. the C++ standard library use them as a way to let users customize operations in certain algorithms. We focus the second part of the talk on examples on how to use Phoenix instead of writing regular function objects and how to enable your legacy code to be used inside Phoenix expressions. However, Phoenix is more. Phoenix is equipped with a unique (in C++) mechanism to handle the expressions discussed in the previous sections as data. This allows us to handle Phoenix not in the C++ standard way but in any way you like. An overview of these mechanisms will be given in the last part of the talk to give potential users an insight on possible future applications that might evolve around Phoenix.

Jun 08

This is the first time I missed attending BoostCon (May 15-20, 2011 – Aspen, Colorado). Fortunately, for us who were not able to attend, Marshall Clow uploaded some videos. Here’s one one that’s relevant to Spirit: “Spirit.Qi in the Real World”, by Robert Stewart. Watch the presentation here:

You can find the slides here:

Past sessions on Spirit have focused on introducing Spirit or showing extracts of real use, intermingled with tutorial highlights. Upon writing real Spirit.Qi parsers, however, one quickly discovers that “the devil is in the details.” There are special cases, tricks, and idioms that one must discover by trial and error or, perhaps, by following the Spirit mailing list, all of which take time and may not be convenient. In this session, we’ll walk through the development of a Spirit.Qi parser for printf()-style format strings. The result will be a replacement for printf() that is typesafe and efficient.

Tagged with:
preload preload preload