{"id":778,"date":"2009-12-21T05:56:56","date_gmt":"2009-12-21T13:56:56","guid":{"rendered":"http:\/\/boost-spirit.com\/home\/?page_id=778"},"modified":"2010-01-17T20:30:05","modified_gmt":"2010-01-18T04:30:05","slug":"the-magical-power-of-attributes-in-spirit-directives-and-non-terminals","status":"publish","type":"page","link":"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/","title":{"rendered":"The Magical Power of Attributes in Spirit &ndash; Directives and Non-terminals"},"content":{"rendered":"<p>In the previous two installments of this article series (<a href=\"http:\/\/boost-spirit.com\/home\/?page_id=713\">The Magical Power of Attributes in Spirit \u2013 Primitives<\/a> and <a href=\"http:\/\/boost-spirit.com\/home\/?page_id=741\">The Magical Power of Attributes in Spirit \u2013 Operators<\/a>) we talked about the attribute handling in different constructs utilized to build parsers and generators with <em>Spirit<\/em>. We will continue this walk through and touch on the remaining parts. Again, I suggest having a look at the first two parts. There are prerequisite information detailed there.<\/p>\n<p>In this article I will talk about directives and\u00a0non-terminals. There is only so much information we can cover in 3 articles, so this will not be exhaustive. The <em><a href=\"http:\/\/www.boost.org\/doc\/libs\/1_41_0\/libs\/spirit\/doc\/html\/index.html\">Spirit documentation<\/a><\/em> is still your friend. Please consult it whenever you are stuck.<\/p>\n<h4>Component Directives<\/h4>\n<p>Directives are special constructs designed to non-intrusively change the behavior of some part of the grammar. The syntax of directives has one of the following forms:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\ndirective[\u2026grammar to modify\u2026]\r\ndirective(param1, param2,\u2026)[\u2026grammar to modify\u2026]\r\n<\/pre>\n<p>The part of the grammar to modify is embedded inside the square bracket. Directives may have additional parameters (here: <span style=\"font-family: Courier New;\">param1<\/span>, <span style=\"font-family: Courier New;\">param2<\/span>, \u2026). In terms of attributes <em>Spirit<\/em> implements three different types of directives:<\/p>\n<ul>\n<li>Directives exposing no attribute (<span style=\"font-family: Courier New;\">unused_type<\/span>), the classical example is qi::omit[] which inhibits parts of the grammar for attribute handling; we will elaborate on this directive later.<\/li>\n<li>Directives exposing the attribute of its embedded components; almost all directives belong to this category, for instance <span style=\"font-family: Courier New;\">qi::nocase[]<\/span>, <span style=\"font-family: Courier New;\">qi::lexeme[]<\/span>, <span style=\"font-family: Courier New;\">karma::left_align[]<\/span>, or <span style=\"font-family: Courier New;\">karma::buffer[]<\/span>. These directives are transparent during attribute handling.<\/li>\n<li>Directives repeating its embedded component, therefore they expose a container holding the attributes of the embedded component; an example is the <span style=\"font-family: Courier New;\">repeat[]<\/span> directive. In terms of its attribute the behavior of <span style=\"font-family: Courier New;\">repeat[]<\/span> is very similar to the Kleene star or the plus operators.<\/li>\n<\/ul>\n<p>The <span style=\"font-family: Courier New;\">omit[]<\/span> directive has interesting properties\u00a0from the standpoint of attributes. If employed as a parser directive, <span style=\"font-family: Courier New;\">omit[]<\/span>inhibits the attribute of the embedded parser. This is very useful when parts of the input do not contribute to the overall required attribute, but the parser for that input exposes some arbitrary attribute anyway. For instance:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nassert(test_parser(qi::omit[qi::char_] &gt;&gt; qi::int_, &quot;x345&quot;, 345));\r\n<\/pre>\n<p><span style=\"font-family: Courier New;\"><span style=\"font-family: Arial;\">In this case the character attribute exposed by <span style=\"font-family: Courier New;\">qi::char_<\/span> does not influence the attribute of the sequence. On the other hand, the <\/span>omit[]<\/span> directive in <em>Karma<\/em> is designed to consume an attribute without emitting any output. The type of the consumed attribute is determined by the embedded generator. Consider an attribute of the type: <span style=\"font-family: Courier New;\">std::pair&lt;int, double&gt;.<\/span> Let us assume we need to emit output from the second member\u00a0only (the <span style=\"font-family: Courier New;\">double<\/span>):<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nstd::pair&lt;int, double&gt; p(1, 2.0);\r\nassert(test_generator(\r\n    karma::omit[karma::int_] &lt;&lt; karma::double_, p, &quot;2.0&quot;));\r\n<\/pre>\n<p>Apart from <span style=\"font-family: Courier New;\">omit[]<\/span>, directives do not exhibit such surprising behavior. Please refer to the corresponding documentation to understand the attribute rules for a particular directive.<\/p>\n<h4>Non-Terminals<\/h4>\n<p>Non-terminals are more interesting. <em>Spirit<\/em> has two officially supported non-terminal types: <span style=\"font-family: Courier New;\">grammars<\/span> and <span style=\"font-family: Courier New;\">rules<\/span> (the <a href=\"http:\/\/www.boost.org\/repository\/doc\/html\/index.html\">Spirit Repository<\/a> additionally has <code>subrules<\/code>, which conform to a similar interface). The <span style=\"font-family: Courier New;\">grammar<\/span> encapsulates a set of rules, primitive parsers and sub-grammars. It is the main mechanism for modularization and composition and they can be composed to form more complex grammars. The <span style=\"font-family: Courier New;\">rule<\/span> is a polymorphic component that acts as a named placeholder capturing the behavior of a <em>Spirit<\/em> expression assigned to it. Both, grammars and rules, are completely identical in terms of attribute handling. For this reason the following section will concentrate on <span style=\"font-family: Courier New;\">rule<\/span>&#8216;s only, leaving the <span style=\"font-family: Courier New;\">grammar<\/span> construct as an exercise for the reader.<\/p>\n<p>Generally, non-terminals are very similar to functions. They may take parameters \u2013 their <em>inherited<\/em> attributes, and they usually return a value \u2013 their <em>synthesized<\/em> attribute. The types of the inherited and the synthesized attributes have to be explicitly specified when defining the particular <code>grammar<\/code> or the <code>rule<\/code>.\u00a0As an example, the following code declares a <em>Qi<\/em> <code>rule<\/code> exposing a <code>double<\/code> as its synthesized attribute, while expecting a <code>std::string<\/code> as its only\u00a0inherited attribute:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nqi::rule&lt;Iterator, double(std::string)&gt; r =\r\n    qi::lit(qi::_r1) &gt;&gt; qi::double_;\r\n<\/pre>\n<p>Three things are worth mentioning here:<\/p>\n<ol>\n<li>The function declaration expression <span style=\"font-family: Courier New;\">double(std::string)<\/span> does not declare a function in the common sense. We use this syntax to specify the attributes in a compact way. The <span style=\"font-family: Courier New;\">double<\/span> is the <em>synthesized<\/em> attribute (as &#8216;returned&#8217; from the rule), while the <span style=\"font-family: Courier New;\">std::string<\/span> is an <em>inherited<\/em> attribute (as it is a &#8216;parameter&#8217; to the rule).<\/li>\n<li>The right hand side expression assigned to the rule always needs to expose an attribute compatible with the rule&#8217;s synthesized attribute. In the example above the <span style=\"font-family: Courier New;\">lit<\/span> does not expose any attribute, while the <span style=\"font-family: Courier New;\">double_<\/span> component exposes a <span style=\"font-family: Courier New;\">double<\/span>. According to the attribute propagation rules for sequences, the attributes of both sides are compatible indeed.<\/li>\n<li>We use yet another set of special, predefined placeholders to access the inherited attributes a rule has been invoked with<span style=\"font-family: Courier New;\">.<\/span> Here <span style=\"font-family: Courier New;\">qi::_r1<\/span> refers to the first inherited attribute of the left hand side rule. Similarily,\u00a0<span style=\"font-family: Courier New;\">qi::_r2<\/span>, <span style=\"font-family: Courier New;\">qi::_r3<\/span>, etc. can be used to access the second, third, etc. inherited attributes, if needed.<\/li>\n<\/ol>\n<p>In the world of generators, non-terminals are just as useful as in the parser world. Generator non-terminals encapsulate a format description for a particular data type, and, whenever we need to emit output for this data type, the corresponding non-terminal is invoked in a similar way as the\u00a0predefined <em>Karma<\/em> generator primitives. <em>Karma<\/em> non-terminals are very similar to the non-terminals in <em>Qi<\/em>. Generator non-terminals may accept parameters as well, and we call those <em>inherited<\/em> attributes too. The main difference is that they do not expose a synthesized attribute (as parsers do), but they require a special <em>consumed<\/em> attribute. Usually the consumed attribute is the value the generator creates its output from. Even if the consumed attribute is not &#8216;returned&#8217; from the generator we chose to use the same function style declaration syntax as employed in <em>Qi<\/em>. The example below defines a <em>Karma<\/em> <code>rule<\/code> consuming a <code>double<\/code> while not expecting any additional inherited attributes.<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nkarma::rule&lt;OutputIterator, double()&gt; r = karma::double_;\r\n<\/pre>\n<p><em>Karma<\/em> non-terminals follow the same rules as outlined for <em>Qi<\/em> non-terminals above. The attributes of the right and left hand sides must be compatible. We did not declare any inherited attributes in the example above, but if we did we would have used <span style=\"font-family: Courier New;\">karma::_r1<\/span>, etc. as the placeholders representing those.<\/p>\n<blockquote><p>It is very important to remember to\u00a0apply the function style declaration syntax while specifying the attributes of a non-terminal. The problem is that non-terminals may take more template parameters than shown above, one of which is the <em>Qi<\/em> skip parser (or the <em>Karma<\/em> delimiting generator). As the template parameters may be passed in any sequence, specifying the attributes using the function style declaration is the only means to identify them. Forgetting to utilize this declaration style might result in difficult to decode compilation errors or in unexpected runtime behavior.<\/p><\/blockquote>\n<p>Inherited attributes for <span style=\"font-family: Courier New;\">rule<\/span>&#8216;s are very similar to expressions we saw earlier. For instance we described the primitive components <span style=\"font-family: Courier New;\">char_(&#8216;a&#8217;)<\/span> or <span style=\"font-family: Courier New;\">lit(&#8220;abc&#8221;)<\/span>, where the <span style=\"font-family: Courier New;\">&#8216;a&#8217;<\/span> and the <span style=\"font-family: Courier New;\">&#8220;abc&#8221;<\/span> are the inherited attributes passed to the <span style=\"font-family: Courier New;\">char_<\/span> and <span style=\"font-family: Courier New;\">lit<\/span> components. The rule defined in the <em>Qi<\/em> example above always needs to be invoked with a <span style=\"font-family: Courier New;\">std::string<\/span> argument. For instance:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nstd::string str(&quot;num: &quot;);\r\nassert(test_parse(r(phoenix::val(str)), &quot;num: 2.0&quot;, 2.0));\r\nstr = &quot;prefix: &quot;;\r\nassert(test_parse(r(phoenix::val(str)), &quot;prefix: 3.1&quot;, 3.1));\r\n<\/pre>\n<p>In this case the inherited attribute defines the prefix to be matched before matching the double. We wrapped the string parameter into a <span style=\"font-family: Courier New;\">phoenix::val()<\/span> because of a problem in the current version of <em>Spirit<\/em> requiring to pass all inherited attributes except scalars as lazy expressions (function objects). This limitation will be removed in future versions of the library.<\/p>\n<h4>Conclusion<\/h4>\n<p><em>Spirit<\/em> has a lot more components than I was able to cover in these three articles. Some of those do not fit into any category, making\u00a0them difficult to describe in a generalized way. If you need more information about a specific component it is always the best to consult the documentation. The quick references (see <em><a href=\"http:\/\/www.boost.org\/doc\/libs\/1_41_0\/libs\/spirit\/doc\/html\/spirit\/qi\/quick_reference.html\">Qi&#8217;s quick reference<\/a><\/em> and <a href=\"http:\/\/www.boost.org\/doc\/libs\/1_41_0\/libs\/spirit\/doc\/html\/spirit\/karma\/quick_reference.html\"><em>Karma&#8217;s<\/em> quick reference<\/a>) give a good overview about what components are available, while each of the components has a separate reference section describing it.<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li><a href=\"#\" class=\"sharing-anchor sd-button share-more\"><span>Share<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><div class=\"sharing-hidden\"><div class=\"inner\" style=\"display: none;\"><ul><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-778\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-778\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-pinterest\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-pinterest-778\" class=\"share-pinterest sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=pinterest\" target=\"_blank\" title=\"Click to share on Pinterest\" ><span>Pinterest<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-778\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-reddit\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-reddit sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=reddit\" target=\"_blank\" title=\"Click to share on Reddit\" ><span>Reddit<\/span><\/a><\/li><li class=\"share-tumblr\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-tumblr sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=tumblr\" target=\"_blank\" title=\"Click to share on Tumblr\" ><span>Tumblr<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>In the previous two installments of this article series (The Magical Power of Attributes in Spirit \u2013 Primitives and The Magical Power of Attributes in Spirit \u2013 Operators) we talked about the attribute handling in different constructs utilized to build parsers and generators with Spirit. We will continue this walk through and touch on the [&hellip;]<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li><a href=\"#\" class=\"sharing-anchor sd-button share-more\"><span>Share<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><div class=\"sharing-hidden\"><div class=\"inner\" style=\"display: none;\"><ul><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-778\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-778\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-pinterest\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-pinterest-778\" class=\"share-pinterest sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=pinterest\" target=\"_blank\" title=\"Click to share on Pinterest\" ><span>Pinterest<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-778\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-reddit\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-reddit sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=reddit\" target=\"_blank\" title=\"Click to share on Reddit\" ><span>Reddit<\/span><\/a><\/li><li class=\"share-tumblr\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-tumblr sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/the-magical-power-of-attributes-in-spirit-directives-and-non-terminals\/?share=tumblr\" target=\"_blank\" title=\"Click to share on Tumblr\" ><span>Tumblr<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div><\/div><\/div>","protected":false},"author":3,"featured_media":0,"parent":422,"menu_order":3,"comment_status":"open","ping_status":"open","template":"article-page.php","meta":{"_s2mail":"","spay_email":""},"jetpack_shortlink":"https:\/\/wp.me\/PIHdZ-cy","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/778"}],"collection":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/comments?post=778"}],"version-history":[{"count":14,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/778\/revisions"}],"predecessor-version":[{"id":782,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/778\/revisions\/782"}],"up":[{"embeddable":true,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/422"}],"wp:attachment":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/media?parent=778"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}