{"id":741,"date":"2009-12-14T08:04:28","date_gmt":"2009-12-14T16:04:28","guid":{"rendered":"http:\/\/boost-spirit.com\/home\/?page_id=741"},"modified":"2010-02-24T05:07:16","modified_gmt":"2010-02-24T13:07:16","slug":"the-magical-power-of-attributes-in-spirit-operators","status":"publish","type":"page","link":"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-operators\/","title":{"rendered":"The Magical Power of Attributes in Spirit – Operators"},"content":{"rendered":"

This is the second in a series of articles giving an introduction to the attribute handling in Spirit<\/em>. It might be a good idea to have a look at the first part before\u00a0reading on as I\u2019m going to rely on what is described there (The Magical Power of Attributes in Spirit \u2013 Primitives<\/a>).\u00a0 In this article I will talk about Spirit\u2019s<\/em> operators. These are being used to combine other components into more complex expressions so as to build more powerful Qi<\/em> parsers and Karma<\/em> generators. We have four different types of operators in Spirit<\/em>, each of which will be covered in more detail below.<\/p>\n

Each operator implements its own set of attribute propagation rules. These rules define the synthesized attributes of Qi<\/em> expressions and the consumed attributes of Karma<\/em> expressions. As we have seen already, it is very important to understand how the synthesized and consumed attributes are formed. Parsers and generators are most elegant and efficient if they are tightly integrated with your data structures, and learning to match parsers to attributes is most crucial to successfully and effectively use Spirit<\/em>.<\/p>\n

This article covers\u00a0 the most important of Spirits<\/em> operators only, so for a full list of all available operators and how the they synthesize or consume attributes please see the documentation sections Parser Compound Attribute Rules<\/a> and Generator Compound Attribute Rules<\/a>.<\/p>\n

Throughout Spirit’s documentation and for the purpose of these articles we use a special notation to describe the attribute propagation rules for compound components. It is of the form:<\/p>\n

a: A, b: B, … \u2013> (composite-expression): composite-attribute<\/span><\/p>\n

where: <\/span>a, b, etc.<\/span> are the operands; A, B, etc.<\/span> are the operand’s attribute types; composite-expression<\/span> is the expression involving the operands; and composite-attribute<\/span> is the resulting attribute type of the composite expression. For instance:<\/p>\n

a: A, b: B \u2013> (a >> b): tuple<A, B><\/span><\/p>\n

which reads as: given, a<\/span> and b<\/span> are components, and A<\/span> is the type of the attribute of a<\/span>, and B<\/span> is the type of the attribute of b<\/span>, then the type of the attribute of a >> b<\/span> will be tuple<A, B><\/span>.<\/p><\/blockquote>\n

All examples shown here are based on the test functions introduced in the first part of this article<\/a>.<\/p>\n

The Optional Operator<\/h4>\n

The optional operator is the simplest of all operators used in Spirit<\/em>. As the name implies, it is used to make some part (or the whole) of a Spirit<\/em> expression optional. For Qi<\/em> parsers the corresponding parts of the input are optional and the whole input will match even if those parts are missing. For Karma<\/em> generators the corresponding data will be emitted only if it is present, therefore adapting the generated output to the available data at runtime. Spirit uses the unary minus (‘-‘<\/span>) as the optional operator. Here are the attribute propagation rules for the optional operator:<\/p>\n\n\n\n\n
Qi<\/span><\/em><\/td>\na: A\u00a0\u2013> (-a): optional<A>
\na: Unused\u00a0\u2013> (-a): Unused<\/span><\/td>\n<\/tr>\n
Karma<\/span><\/em><\/td>\na: A\u00a0\u2013> (-a): optional<A>
\na: Unused\u00a0\u2013> (-a): Unused
\n<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n

The optional<A><\/span> in the table above is a placeholder for boost::optional<A><\/span>. Unused<\/code> stands for unused_type<\/code>, which is just Spirits way of saying \u2018I don\u2019t care\u2019. In this context it is interpreted as if the component does not expose any attribute at all. The result of executing an optional parser expression will be either a non-initialized boost::optional<A><\/span> if the embedded parser did not match, or it will hold its returned attribute if it did match. An optional generator expression will emit the supplied attribute only if the boost::optional<A><\/span> has been properly initialized, otherwise no output will be generated.<\/p>\n

The following examples parse an optional integer showing it succeeds even if no integer is found in the input. The examples also demonstrates the generation of an optional integer, again, always succeeding, even if no integer is passed to the generator.<\/p>\n

\r\nassert(test_parser(-qi::int_, "", boost::optional<int>()));\r\nassert(test_parser(-qi::int_, "1234", boost::optional<int>(1234)));\r\nassert(test_generator(-karma::int_, boost::optional<int>(), "")));\r\nassert(test_generator(-karma::int_, boost::optional<int>(1234), "1234")));\r\nassert(test_generator(-karma::int_, 1234, "1234")));\r\n<\/pre>\n
As the last line shows, even if the consumed attribute of the Karma<\/em> operator is a boost::optional<\/span>, it is still compatible with any non-optional attribute as long as it is convertible to the consumed attribute of the embedded generator.<\/p>\n
Repetition Operators<\/h4>\n
Spirit<\/em> has several repetition operators, such as the Kleene star (unary ‘*’<\/span>), the plus (unary ‘+’<\/span>) operator, and lists (‘%’<\/span>), etc.. All of them are similar as they handle several elements of the same type. In terms of attribute handling, they are all equivalent. The following table shows the attribute rules based on those for the Kleene star operator, but all others are similar.<\/p>\n\n\n\n\n
Qi<\/span><\/em><\/td>\na: A\u00a0\u2013> (*a): vector<A>
\na: Unused\u00a0\u2013> (*a): Unused<\/span><\/td>\n<\/tr>\n
Karma<\/span><\/em><\/td>\na: A\u00a0\u2013> (*a): vector<\/span><A>
\na: Unused \u2013> (*a): Unused
\n<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
The vector<A><\/span> is used as a placeholder for any standard container (it is possible to adapt your own data structures). Unused<\/code> stands for unused_type<\/code>. In short, repetitive parsers directly fill any supplied standard container as long as the value type of the container is compatible with the synthesized attribute of the embedded parser. Likewise, repetitive generators directly use the elements of any supplied standard container, one element for each invocation of the embedded generator.<\/p>\n
\r\nstd::vector<char> v;\r\nv.push_back('a'); v.push_back('b'); v.push_back('c');\r\nassert(test_parser(*qi::char_, "abc", v));\r\nassert(test_generator(*karma::char_, v, "abc")));\r\n<\/pre>\n
Sequence Operators<\/h4>\n
Sequences are arguably the most powerful and useful of all operators but, at the same time, their attribute handling is the most difficult to fully understand. Sequence operators are used to build, well, sequences of other components. In order to build a sequence in Qi<\/em> we use the ‘>>’<\/span> operator. In Karma<\/em> the ‘<<‘<\/span> operator is used. This is probably not surprising as it corresponds to the conventions used for C++ streams, where those operators are used for input (‘>>’<\/span>) and output (‘<<‘<\/span>) as well. Here are the main attribute propagation rules forsequences:<\/p>\n\n\n\n\n
Qi<\/span><\/em><\/td>\na: A, b: B\u00a0\u2013> (a >> b): tuple<A, B>
\na: A, b: Unused\u00a0\u2013> (a >> b): A<\/span><\/td>\n<\/tr>\n
Karma<\/span><\/em><\/td>\na: A, b: B\u00a0\u2013> (a << b): tuple<A, B>
\na: A, b: Unused\u00a0\u2013> (a << b): A
\n<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Again, the tuple<A, B, \u2026><\/span> in the table above is used as a placeholder only. The notation stands for any Boost.Fusion<\/a> sequence<\/em> holding elements of types A<\/code>, B<\/code>, … etc. Unused<\/code> stands for unused_type<\/code>.<\/p>\n
Sequences in Boost.Fusion<\/em> are a generalization of tuples, which includes Fusion\u2019s<\/em> own data structures (such as fusion::vector<\/a><\/span>, fusion::list<\/a><\/span>, etc.), Fusion\u2019s views<\/a>, std::pair<\/span>, boost::array<\/a><\/span>, and any C++ struct (although this requires to use the macro BOOST_FUSION_ADAPT_STRUCT<\/a><\/span>). Especially the ability to \u2018map\u2019 a plain C++ struct onto a Spirit<\/em> component sequence is a very powerful concept enabling tight integration of parsing and generation with your own (arbitrary) data structures.<\/p><\/blockquote>\n
The following example parses two comma separated double<\/span> numbers and outputs those in reverse order:<\/p>\n
\r\nassert(test_parser(qi::double_ >> ',' >> qi::double_,\r\n    "1.0,2.0", std::make_pair(1.0, 2.0)));\r\nassert(test_generator(karma::double_ << ',' << karma::double_,\r\n    std::make_pair(2.0, 1.0), "2.0,1.0")));\r\n<\/pre>\n
We use a std::pair<double, double><\/span> as the attribute for our sequence (std::pair<\/span> perfectly conforms to the notion of a Fusion<\/em> sequence). The literal ‘,’<\/span> does not expose any attribute (Unused<\/span>), which makes it \u2018disappear\u2019 from the attribute handling (see the related attribute propagation rules in the table above). As a result, the member variable first<\/span> of the pair is used as the attribute for the first double<\/span> in the sequence,\u00a0 while the pair\u2019s member variable second <\/span>is the used as the attribute for the second double<\/span> in the corresponding Spirit expression.<\/p>\n
Qi<\/em> and Karma<\/em> expose a set of API functions usable with sequences. Very much like the functions of the scanf<\/code> and printf<\/code> families these API functions allow to pass the attributes for each of the elements of the sequence separately. Using the corresponding overload of Qi’s<\/em> parse()<\/span> or Karma’s<\/em> generate()<\/code> the expression above could be rewritten as:<\/p>\n
\r\ndouble d1 = 0.0, d2 = 0.0;\r\nqi::parse(f, l, qi::double_ >> ',' >> qi::double_, d1, d2);\r\nkarma::generate(sink, karma::double_ << ',' << karma::double_, d2, d1);\r\n<\/pre>\n
where the first argument is used for the first double_<\/code>, and the second argument is used for the second double_<\/code>. This provides a clear and comfortable syntax, more similar to the placeholder based syntax as exposed by printf<\/code> or boost::format<\/code>.<\/p>\n
It is possible to pass a (standard) container as an attribute for a Spirit<\/em> sequence if all elements of that sequence expose either the container or its value type as their attributes. So we could rewrite the example above by replacing the std::pair<double, double><\/span> with any standard container type, such as std::vector<double><\/span>. This feature has been added to ensure full semantic compatibility of repetitive constructs like double_ % ‘,’<\/span> with the equivalent sequence expression double >> *(‘,’ >> double_)<\/span>, but has been proven to be useful in general.<\/p>\n
We have seen in the first article about primitives how the special placeholder _1<\/span> can be used in a semantic action to refer to the attribute of the expression it is attached to. But wait, there is more! If you attach a semantic action to a sequence, each of the attributes of the elements can be accessed using the corresponding placeholders _2<\/span>, _3<\/span>, etc.:<\/p>\n
\r\n(qi::double_ >> ',' >> qi::double_)\r\n    [ std::cout << qi::_1 << ',' << qi::_2 ]\r\n(karma::double_ << ',' << karma::double_)\r\n    [ karma::_1 = 1.0, karma::_2 = 2.0 ]\r\n<\/pre>\n
The first expression will print the two matched doubles separated with a comma, while the second expression will emit “1.0,2.0″<\/span>. This is consistent with the multi-attribute API functions mentioned above.<\/p>\n
The Alternative Operator<\/h4>\n
Alternatives in Spirit<\/em> are built using the ‘|’<\/span> operator. Parser alternatives are used to specify different possible valid input formats to be matched alternatively. The result of an alternative match are attributes with possibly different types. Each of the alternatives may potentially expose its own attribute type. Generator alternatives are used to specify different output formats to apply depending on the type of the supplied attribute. As a logical consequence the attribute propagation rules for the alternative operator are:<\/p>\n\n\n\n\n
Qi<\/span><\/em><\/td>\na: A, b: B\u00a0\u2013> (a | b): variant<A, B>
\na: A, b: A\u00a0\u2013> (a | b): A
\na: A, b: Unused\u00a0\u2013> (a | b): optional<A><\/span><\/td>\n<\/tr>\n
Karma<\/span><\/em><\/td>\na: A, b: B\u00a0\u2013> (a | b): variant<A, B>
\na: A, b: Unused\u00a0\u2013> (a | b): optional<A>
\n<\/span><\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n
Here the variant<A, B><\/span> stands for a boost::variant<A, B><\/span> and the optional<A><\/span> is a placeholder for boost::optional<A><\/span>. Please note how Unused<\/span> is handled in a special way. It disappears from the attribute handling again, but all other alternatives will get optional. If all alternatives expose the same attribute type, the overall attribute will be the same. Let us have a look at an example:<\/p>\n
\r\nusing boost::variant;\r\ntest_parser(qi::int_ | qi::bool_, "1234", variant<bool, int>(1234));\r\ntest_parser(qi::int_ | qi::bool_, "true", variant<bool, int>(true));\r\ntest_generator(karma::int_ | karma::string, 4321, "4321"));\r\ntest_generator(karma::int_ | karma::string, std::string("a"), "a"));\r\n<\/pre>\n
The alternative parser simply fills the variant with the appropriate attribute type based on the matched input. The generator alternative has the additional feature of doing the selection of the output format based on the supplied attribute type: data driven format selection at work! The generator will choose the first matching alternative for the supplied attribute. You don\u2019t even have to pass a variant to the generator (although, you certainly can), any attribute compatible with any of the types of the alternatives can be used, causing the corresponding formatting expression to be chosen.<\/p>\n
Conclusion<\/h4>\n
Building Spirit<\/em> expressions is very similar to building normal C++ expressions: the types of the right hand side and the left hand side must be compatible. The difference is that for Spirit<\/em> expressions you have to think about the attribute types of the parts of the expression, not the C++ types of those. Once you understand the attribute propagation rules of the different operators it will be a lot easier to build Qi<\/em> parsers and Karma<\/em> generators which are compatible with your own data types. Most of the time you should be able to integrate your data without having to resolve to semantics actions. This does not only make your code simpler and more readable, it has the additional benefit of being faster, both at compile time and at runtime.<\/p>\n
Share this:<\/h3>