Welcome to Spirit

Spirit is a set of C++ libraries for parsing and output generation implemented as Domain Specific Embedded Languages (DSEL) using Expression templates and Template Meta-Programming. The Spirit libraries enable a target grammar to be written exclusively in C++. Inline grammar specifications can mix freely with other C++ code and, thanks to the generative power of C++ templates, are immediately executable.

Spirit is part of Boost Libraries, a peer-reviewed, open collaborative development effort.

Aug ’13 04

A customization point is a code construct that the user can leverage to specialize how a particular library action is handled. A common way of implementing this in C++ is to define a template with the default behavior, and let users specialize it for their own types —e.g., std::hash—. This is the story of Spirit X3 and how it lets you specialize customization points without ever leaving your own namespace, sort of…

I Will Always Find You – Tales of C++


Jun ’11 08

Here’s another BoostCon video uploaded by Marshall Clow. This one is about Phoenix V3, by Hartmut Kaiser:


The slides for this talk can be found here: https://github.com/boostcon/2011_presentations/blob/master/mon/phoenix_v3.pdf?raw=true.

Phoenix will be the next generation of creating unnamed, inlined polymorphic function objects. With V3 we combine the functionality of Boost.Bind and Boost.Lambda, and arranges it into a new library. By writing this new library, we were able to fix some limitations of the aforementioned libraries without breaking backwardscompatibility. The purpose of the talk will be to outline the importance and elegance of functional programming (FP) in C++. The first part of the talk will give an introduction into the Domain Specific Embedded Language (DSEL) we defined with Phoenix. A DSEL is built with the help of regular C++ function and operator overloads. For Phoenix we defined such a language that emulates C++, to give potential users a low entry into the world of FP. While a lot of existing C++ code relies on higher order functions (better known as function objects), e.g. the C++ standard library use them as a way to let users customize operations in certain algorithms. We focus the second part of the talk on examples on how to use Phoenix instead of writing regular function objects and how to enable your legacy code to be used inside Phoenix expressions. However, Phoenix is more. Phoenix is equipped with a unique (in C++) mechanism to handle the expressions discussed in the previous sections as data. This allows us to handle Phoenix not in the C++ standard way but in any way you like. An overview of these mechanisms will be given in the last part of the talk to give potential users an insight on possible future applications that might evolve around Phoenix.

Jun ’11 08

This is the first time I missed attending BoostCon (May 15-20, 2011 – Aspen, Colorado). Fortunately, for us who were not able to attend, Marshall Clow uploaded some videos. Here’s one one that’s relevant to Spirit: “Spirit.Qi in the Real World”, by Robert Stewart. Watch the presentation here:


You can find the slides here: https://github.com/boostcon/2011_presentations/raw/master/tue/spirit_qi_in_the_real_world.pdf

Past sessions on Spirit have focused on introducing Spirit or showing extracts of real use, intermingled with tutorial highlights. Upon writing real Spirit.Qi parsers, however, one quickly discovers that “the devil is in the details.” There are special cases, tricks, and idioms that one must discover by trial and error or, perhaps, by following the Spirit mailing list, all of which take time and may not be convenient. In this session, we’ll walk through the development of a Spirit.Qi parser for printf()-style format strings. The result will be a replacement for printf() that is typesafe and efficient.

Tagged with:
Feb ’11 28

When using expectation points, a parsing failure results in an exception that generically indicates the failure, but probably doesn’t explain the problem in the most meaningful way. It is possible to attach an error handler to react to the failed match in a more specialized way:

Continue reading »

Feb ’11 12

Narinder Claire asked a seemingly innocent question on the Spirit mailing list the other day. After starting to write an answer I realized that this question is not innocent at all as it touches the very fabric of Spirit: the rules of attribute handling. Many people have a hard time to properly understand what is going on in the nether regions of Spirit. More importantly, they have a hard time to understand why is Spirit implemented the way it is.

The new article Attribute Propagation and Attribute Compatibility not only answers Narinders questions but tries to explain those important concepts in more detail.

Tagged with:
Dec ’10 03

These are links to the slides and video of Michael Caisse’s BoostCon 2010 talk:

slides: <http://www.objectmodelingdesigns.com/boostcon10/>
video: <http://blip.tv/file/4143337 >


Machinery, sensors, equipment, client/server communications, even file formats… Parsing and producing communication streams is everywhere you look. Often these tasks are simple or small enough to tempt ad-hoc solutions. The Spirit 2.1 library provides a model that is simple enough to tackle those “quick hacks” and easily scales for full-featured AST generation.

This session will explore real-life experiences with the parser and generator (Qi/Karma) portions of the Spirit library. As we look at various small and medium-sized parsers/generators employed in various products we will establish some “rules-of-thumb” and guidelines for tackling the parser/generator domain with Qi/Karma. The session will end with the implementation of a usable XML parser and a simplified XPath-like node extractor.

The session will include some lecture and a lot of tutorial. Attendees will walk away with the knowledge and tools to begin parsing and generating with Spirit Qi/Karma.

—Michael Caisse

Tagged with:
Nov ’10 13

Jeroen Habraken (a.k.a VeXocide) sent an article about parsing escaped strings using Qi, which we happily publish for everybody to read. Thanks Jeroen!

Continue reading here.

Tagged with:
Oct ’10 14

Care about how Spirit got started? Here’s a link to our BoostCon 2010 presentation:


This year, we celebrate Spirit’s 10th anniversary from its early beginnings as an offshoot from a much larger GUI library in the 90s and debuted into Boost in May 2001 in the typical “Is there interest in this library?” fashion like all would be Boost libraries. From a humble 7 header file library, Spirit has grown to be one of the most sophisticated Boost libraries and along the way became the incubator of other Boost libraries such as Boost.Fusion, Boost.Phoenix, and Boost.Wave and played a significant role for Boost.Proto getting mature.
We would like to present Spirit (and the libraries it inspired) in a historical perspective. The presentation will aim to provide a lighter, more intimate perspective into the development of at least 4 libraries with almost a decade’s worth of experience being Boost authors and bonafide crazy template metaprogrammers who abuse operators like Mad Scientists. Of course, we can’t help it if we show off some C++ tricks here and there, but we’ll try to keep it as light as we can.

Tagged with:
Mar ’10 05

Tracking the Input Position While Parsing

By Peter Schüller Advanced, Beginner, Qi Example Comments Off on Tracking the Input Position While Parsing

The following article is about tracking the parsing position with Spirit V2. This is useful for generating error messages which tell the user exactly where an error has occurred. We also show how to use Spirit V2 to parse from an input stream without first reading the whole stream into a std::string.

Continue reading »

Tagged with:
Mar ’10 03

The concept of Spirit’s semantic actions seems to be easy enough to understand as most people new to the library prefer their usage over applying the built-in attribute propagation rules. That is not surprising. The idea of attaching a function to any point of a grammar which is called whenever the corresponding parser matched is straighforward to grasp. Earlier versions of Spirit required a semantic action to conform to a very specific interface. Today’s semantic actions are more flexible and more powerful. Recently, a couple of people asked questions about them. So I decided dedicating this Tip of the Day to the specifics and the usage model of semantic actions in Spirit Qi.

Continue reading »

Tagged with:
Feb ’10 24

Spirit supports skipper based parsing since its very invention. So this is definitely not something new to Spirit V2. Nevertheless, the recent discussion on the Spirit mailing list around the semantics of Qi’s lexeme[] directive shows the need for some clarification. Today I try to answer questions like: “What does it mean to use a skipper while parsing?”, or “When do I want to use a skipper and when not?”.

Continue reading »

Tagged with:
Feb ’10 17

Recently, there have been a couple of questions on the Spirit mailing list asking how to parse as set of things known in advance in any sequence and any combination. A simple example would be a list of key/value pairs with known keys but the keys may be ordered in any sequence. This use case seems to be quite common. Fortunately Spirit provides you with a predefined parser component designed for exactly that purpose: the permutation parser.

Continue reading »

Tagged with:
Feb ’10 08

Here is another question raised from time to time: “I know how to use a plain struct as an attribute for a sequence parser in Qi by adapting it with BOOST_FUSION_ADAPT_STRUCT. Unfortunately this does not work if the struct is a template. What can I do in this case?”.

There have been plans for a while to create a separate Fusion facility BOOST_FUSION_ADAPT_TPL_STRUCT allowing to adapt templated data types, but this is not in place yet. Today I will describe a trick you can apply to adapt your templates into ‘proper’ Fusion sequences anyway.

Continue reading »

Tagged with:
Jan ’10 31

Most questions on the main Spirit mailing list are centered around attributes and how to correctly utilize Spirit’s attribute propagation rules. We discussed the related basics in several posts already, but many people still have problems to understand the rules. Somebody recently asked on the mailing list whether it would be possible to print the attribute type exposed by an arbitrary parser expression. I answered by posting a sketchy code snippet (see here), but afterwards I realized it might be a good topic for yet another ‘Tip of the Day’.

Continue reading »

Tagged with:
Jan ’10 28

After writing about local variables for rules here I would like to get back to some information I have had lying around for some time already. Today’s topic seems to be a nice fit as porting the dynamic parsers from Spirit.Classic requires to utilize local variables. This allows me to give you some more examples for this facility.

Previous versions of Spirit, which are the versions we refer to as Spirit.Classic today, implemented special dynamic parsers allowing to insert control statements into the parsing process (such as if_p, while_p, and for_p). These dynamic parsers are not available anymore in Qi. But it is easy enough to achieve the same behavior using existing Qi components. This ‘Tip of the Day’ describes those techniques, which have been developed and contributed by Carl Barron. Thanks Carl!

Continue reading »

Tagged with:
preload preload preload