Welcome to Spirit

Spirit is a set of C++ libraries for parsing and output generation implemented as Domain Specific Embedded Languages (DSEL) using Expression templates and Template Meta-Programming. The Spirit libraries enable a target grammar to be written exclusively in C++. Inline grammar specifications can mix freely with other C++ code and, thanks to the generative power of C++ templates, are immediately executable.

Spirit is part of Boost Libraries, a peer-reviewed, open collaborative development effort.

Jul 23

Thomas Heller has just completed the Phoenix Proto port. Another successful GSOC project!

I’ve been mentoring for GSOC for a couple of years now. This one is the best (the Fusion 0x project of last year’s GSOC was also highly successful, but it was Hartmut Kaiser who mentored that). Thomas is an amazing student! A++, if there’s such a thing.

Here’s Thomas’ post on the Boost mailing list:

Ladies and Gentlemen,

I proudly announce that the port of phoenix3 is completed! All  testcases pass! (some with minor modifications)

So, What is next? Here is my proposed schedule:

  • fix some minor issues
  • improve boost.bind compatibility: make all boost.bind testcases pass
  • implement all boost.lambda testcases and make them pass, as far as it is reasonable
  • add support for C++0x lambdas
  • make interoperability testcases for std::function, boost::function
  • adapt the documentation
  • clean up code (some parts are a little messy as of now)
  • improve compile time

Did I miss something?

This is a very tight schedule, I may not be able to finish all points during the GSoC period. Any help will be appreciated. Additionally a review of the current code is highly appreciated to point out some defects that might exist.

The code can be checked out at:
https://svn.boost.org/svn/boost/sandbox/SOC/2010/phoenix3

I will be happy to answer your questions!

Additionally I would like to thank Joel de Guzman and Eric Niebler for their great assistance and initial designs!

So why is this relevant? Here’s more from Eric Niebler:

Let me expand a bit on Thomas’ post with my own perspective. Currently, the expressions created by Boost.Bind, Boost.Lambda and Boost.Phoenix2 are black boxes. You can pass them to std algorithms for evaluation, but that’s about it.

In contrast, the expressions created by Boost.Phoenix3 will the Proto expressions. If Phoenix3 is the compiler of a C++-in-C++ domain specific language, then the Proto expression is the intermediate form. It will be documented and part of the Phoenix3 API. The grammar for valid Phoenix3 expressions will also be documented and extensible. For the first time, we will have a way to generate C++-like expression trees, just like the C++ compiler itself does. We have a standard way (Proto) to traverse and manipulate them. Sure, you can just evaluate them as you could before, but now you can do much more. For instance, you can define your own Proto transforms to:

  • Do various optimizations, just like a real compiler
  • Add your own custom evaluation strategies for operations on your types
  • Rewrite Phoenix3 expressions for parallel execution, or whatever
  • ???!!!

Essentially, it means Phoenix3 is a white box, an open platform. Third parties can use just the Phoenix3 front end and intermediate form, substituting their own back ends to make the expressions mean and do completely different and domain-specific things.

Expect to see Phoenix3 expressions showing up in other DSEL contexts. It’s hard to predict how people will use this. The possibilities are really limitless.

Jun 23

We have a new page enumerating the best practices for using Spirit. Right now, there’s only one entry (update: there may be more by the time you read this), but as time goes by, we will be filling this page with more. If you have a short Spirit tip to share, this is the page. Add a comment and we’ll incorporate them.

See the “Best Practices” page here.

Jun 18

Spirit-shirt frontAwesomeness…

Michael Caisse of Object Modeling Designs designed this cool Spirit inspired T-shirt. I love it so much. If you like Spirit as much as we do, you can get one from Michael here: http://objectmodelingdesigns.com/swag.html

May 16

Yes, we were tempted to call our second BoostCon talk ‘Design and Evolution’, but we decided to to be modest. We wanted to give a very brief overview of the way Spirit has evolved over the last decade. While talking about a period of time this long in 90 minutes has to be very brief we nevertheless tried to highlight the main stepping stones and milestones in Spirits development. For me, developing the slides for ‘Spirit: History and Evolution’ was real fun: looking through ancient email archives, re-reading key private conversations, and realizing that most of the ideas implemented today have been already in place from the early years. All of this was exciting. To some extent, this even turned out to be educational.

I have to admit I was not sure how people would react to this kind of talk. It was not one of the usual, sophisticated, well thought through, and cutting edge presentations normally seen at BoostCon. This talk was merely a conversation with the attendees about Boost’s history guided by Spirits evolution. In the end I was relieved to see people reacted positively. Some came to me afterwards saying they enjoyed listening. For all of you not being at BoostCon, here are the slides: Spirit: History and Evolution.

Tagged with:
May 13

Michael Caisse (of Object Modeling Designs), some of you will know him from the Freenode #boost IRC channel, presented his talk about Spirit at BoostCon 2010 last Tuesday. I personally very much like listening to his knowledgeable and entertaining presentations. Apparently many people do so as well, the room he presented in was overfull. People even sat on the floor as all seats were taken. So in case you are interested – here is a link to his slides: Using Spirit V2: Qi and Karma.

Tagged with:
May 11

Finally, yesterday was the day! Joel and I presented the current status of our work related to the ‘Writing a Compiler’ effort. We submitted this talk to BoostCon 2010 and got accepted back in January. Having to present at BoostCon just put a nice pressure onto us to get moving and to start working on the project. I don’t want to go into any detail about the presentation itself. This will be done in future articles to be published after BoostCon. I just wanted to provide a link to the slides for those interested in having a look. Ok, here we go: A Framework for RAD Spirit, grab it while it’s hot!

Tagged with:
May 08

Finally, the long wait and the preparations for BoostCon 2010 are over. The last weeks were dominated by writing the code and creating the slides for our talk ‘A Framework for RAD Spirit‘. Regular readers of this site probably have already wondered why we have been inactive here since mid-March. Well, that is mainly because of preparing for BoostCon turned out to be too time consuming to leave any room for writing posts and articles.

Continue reading »

Tagged with:
Mar 11

I have a mixed relationship with variant…

I just wrote a parser for S-expressions (that will be the basis of ASTs and intermediate types in my planned “write-a-compiler” article series). The parser itself is easy, but as always, I spent more time on the underlying data structures.

Continue reading »

Mar 09

Tino Didriksin published yet another speed comparison of different ways to convert an integer into a string (see his blog A howl on the Wind…). And again, Karma turns out to be the leader of the pack by being upto 10 times faster than the other tested methods. This nicely supports our own measurements (see here). Thanks Tino!

Tagged with:
Mar 05

The following article is about tracking the parsing position with Spirit V2. This is useful for generating error messages which tell the user exactly where an error has occurred. We also show how to use Spirit V2 to parse from an input stream without first reading the whole stream into a std::string.

Continue reading »

Tagged with:
Mar 03

The concept of Spirit’s semantic actions seems to be easy enough to understand as most people new to the library prefer their usage over applying the built-in attribute propagation rules. That is not surprising. The idea of attaching a function to any point of a grammar which is called whenever the corresponding parser matched is straighforward to grasp. Earlier versions of Spirit required a semantic action to conform to a very specific interface. Today’s semantic actions are more flexible and more powerful. Recently, a couple of people asked questions about them. So I decided dedicating this Tip of the Day to the specifics and the usage model of semantic actions in Spirit Qi.

Continue reading »

Tagged with:
Feb 24

Spirit supports skipper based parsing since its very invention. So this is definitely not something new to Spirit V2. Nevertheless, the recent discussion on the Spirit mailing list around the semantics of Qi’s lexeme[] directive shows the need for some clarification. Today I try to answer questions like: “What does it mean to use a skipper while parsing?”, or “When do I want to use a skipper and when not?”.

Continue reading »

Tagged with:
Feb 17

Recently, there have been a couple of questions on the Spirit mailing list asking how to parse as set of things known in advance in any sequence and any combination. A simple example would be a list of key/value pairs with known keys but the keys may be ordered in any sequence. This use case seems to be quite common. Fortunately Spirit provides you with a predefined parser component designed for exactly that purpose: the permutation parser.

Continue reading »

Tagged with:
Feb 08

Here is another question raised from time to time: “I know how to use a plain struct as an attribute for a sequence parser in Qi by adapting it with BOOST_FUSION_ADAPT_STRUCT. Unfortunately this does not work if the struct is a template. What can I do in this case?”.

There have been plans for a while to create a separate Fusion facility BOOST_FUSION_ADAPT_TPL_STRUCT allowing to adapt templated data types, but this is not in place yet. Today I will describe a trick you can apply to adapt your templates into ‘proper’ Fusion sequences anyway.

Continue reading »

Tagged with:
Feb 05

Promising to be the main face-to-face event for all things Boost, BoostCon 2010 opens the door to your C++ future. From using the Boost libraries to writing and maintaining them, from evangelizing to deploying Boost within your organization, from infrastructure and process to vision and mission, and from TR1 to TR2, BoostCon brings together the sessions, the colleagues, and the inspiration to support your work with Boost for the next year.

To reflect the breadth of the Boost community, the conference includes sessions aimed at two constituencies: Boost end-users and hard-core Boost library and tool developers. The program fosters interaction and engagement within and across those two groups, with an emphasis on hands-on, participatory sessions.

Continue reading »

preload preload preload