slides: <http://www.objectmodelingdesigns.com/boostcon10/>
video: <http://blip.tv/file/4143337 >

Enjoy!

Machinery, sensors, equipment, client/server communications, even file formats… Parsing and producing communication streams is everywhere you look. Often these tasks are simple or small enough to tempt ad-hoc solutions. The Spirit 2.1 library provides a model that is simple enough to tackle those “quick hacks” and easily scales for full-featured AST generation.

This session will explore real-life experiences with the parser and generator (Qi/Karma) portions of the Spirit library. As we look at various small and medium-sized parsers/generators employed in various products we will establish some “rules-of-thumb” and guidelines for tackling the parser/generator domain with Qi/Karma. The session will end with the implementation of a usable XML parser and a simplified XPath-like node extractor.

The session will include some lecture and a lot of tutorial. Attendees will walk away with the knowledge and tools to begin parsing and generating with Spirit Qi/Karma.

—Michael Caisse

http://blip.tv/file/4245756

This year, we celebrate Spirit’s 10th anniversary from its early beginnings as an offshoot from a much larger GUI library in the 90s and debuted into Boost in May 2001 in the typical “Is there interest in this library?” fashion like all would be Boost libraries. From a humble 7 header file library, Spirit has grown to be one of the most sophisticated Boost libraries and along the way became the incubator of other Boost libraries such as Boost.Fusion, Boost.Phoenix, and Boost.Wave and played a significant role for Boost.Proto getting mature.
We would like to present Spirit (and the libraries it inspired) in a historical perspective. The presentation will aim to provide a lighter, more intimate perspective into the development of at least 4 libraries with almost a decade’s worth of experience being Boost authors and bonafide crazy template metaprogrammers who abuse operators like Mad Scientists. Of course, we can’t help it if we show off some C++ tricks here and there, but we’ll try to keep it as light as we can.

I have to admit I was not sure how people would react to this kind of talk. It was not one of the usual, sophisticated, well thought through, and cutting edge presentations normally seen at BoostCon. This talk was merely a conversation with the attendees about Boost’s history guided by Spirits evolution. In the end I was relieved to see people reacted positively. Some came to me afterwards saying they enjoyed listening. For all of you not being at BoostCon, here are the slides: Spirit: History and Evolution.

Our  talk (Joel’s and mine) is about the current status and the preliminary results we achieved while working on the compiler project announced earlier. We settled for applying a Scheme based framework to create a rapid development environment for Spirit parsers. This is a really exciting project which will have long term impact not only on Spirit.

Here is a sneak preview of what we will present at BoostCon.

As you can see, we created a kernel consisting out of a cool data structure called U-tree (essentially a discriminated union), capable of storing any  S-expression. On top of this Joel developed a Scheme compiler/interpreter which first constructs a dynamic expression tree from the S-expression (the compiler) conforming to the usual Scheme execution rules. Executing this expression tree yields the result encoded in the Scheme code which has been stored in the S-expression in the first place (the interpreter).

As a byproduct of this we got the beginnings of a new library – the dynamic counterpart of Boost.Phoenix. It is aimed at dynamic execution trees based on functional programming paradigms. So the Scheme compiler creates a functional data structure which is indistinguishable from an equivalent data structure created from a C++ expression built on top of this new library.

In order to fill the S-expression from Scheme code we developed a Spirit based parser. OTOH, the Scheme generator (written using Karma – what else?) takes any S-expression and converts it back to the equivalent Scheme code.

The (currently) top most layer of the framework consists of a Parser for Qi expressions which creates an S-expression and a corresponding generator converting any S-expression holding a Qi parser back into the equivalent Qi (C++) expression. Naturally, the Qi parser is written using Qi and for the Qi Generator we utilized Karma. The Qi compiler mentioned in the figure takes such an S-expression and produces an execution tree, which – when actually run – parses any input conforming to the initial Qi grammar.

I don’t want to spoil the fun and tell too much at this point, but overall this is not only a nice example of how to write more complex Spirit applications, but it gives you a rapid development platform allowing to quickly test Qi parsers. Moreover, as the whole thing is based on a Scheme kernel, you can write your own Scheme code for additional transformations of the handled S-expressions. In the context of the Qi based S-expressions mentioned above this is equivalent to dynamic parser transformations or arbitrary parser analysis modules. That is pretty powerful stuff!

Joel and I plan to write about what we developed and what we learnt while developing this and we hope to get some of you interested to chime in. There is still a lot to do, we are by no means done with this framework. And, as I mentioned already, I believe this will have an impact far beyond its initial target audience.

But BoostCon is not only about Spirit (even if there will be at least 3 talks about it). We will have a full week packed with sessions presented by well known Boost people speaking about a whole set of interesting topics. Just have a look at the BoostCon schedule to convince yourself. Over the next week I will try to document some of my impressions live from the Physics Center in Aspen (where BoostCon takes place). So stay tuned!