{"id":1334,"date":"2011-02-12T16:27:43","date_gmt":"2011-02-13T00:27:43","guid":{"rendered":"http:\/\/boost-spirit.com\/home\/"},"modified":"2011-05-27T12:13:49","modified_gmt":"2011-05-27T19:13:49","slug":"attribute-propagation-and-attribute-compatibility","status":"publish","type":"page","link":"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/","title":{"rendered":"Attribute Propagation and Attribute Compatibility"},"content":{"rendered":"<p>Narinder Claire asked a seemingly innocent question on the <em>Spirit<\/em> mailing list the other day. After starting to write an answer I  realized that this question is not innocent at all as it touches the  very fabric of <em>Spirit<\/em>: the rules of attribute handling. Many  people have a hard time to properly understand <strong>what<\/strong> is going on  in the nether regions of <em>Spirit<\/em>. More importantly, they have a  hard time to understand <strong>why<\/strong> is <em>Spirit<\/em> implemented the way  it is.<img title=\"More...\" src=\"http:\/\/boost-spirit.com\/home\/wp-includes\/js\/tinymce\/plugins\/wordpress\/img\/trans.gif\" alt=\"\" \/><img title=\"More...\" src=\"http:\/\/boost-spirit.com\/home\/wp-includes\/js\/tinymce\/plugins\/wordpress\/img\/trans.gif\" alt=\"\" \/><\/p>\n<p>The question was (I&#8217;m paraphrasing to fit it into this article): the  following code compiles and runs fine:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nstd::map&lt;std::string, std::string&gt; the_dictionary;\r\nvoid insert(std::pair&lt;std::string, std::string&gt;&amp; p)\r\n{\r\n    the_dictionary[p.first] = p.second;\r\n}\r\n\r\nnamespace qi = boost::spirit::qi;\r\n\r\ntypedef std::string::iterator iterator;\r\nqi::rule&lt;iterator, std::string()&gt; identifier = qi::alpha &gt;&gt; *qi::alnum;\r\nqi::rule&lt;iterator, std::pair&lt;std::string, std::string&gt;()&gt; assignment;\r\nassignment = identifier &gt;&gt; '=' &gt;&gt; identifier &gt;&gt; ';';\r\n\r\nstd::string input(&quot;a=b&quot;);\r\nqi::parse(input.begin(), input.end(), assignment[&amp;insert]);\r\n<\/pre>\n<p>It parses the input &#8216;a=b&#8217; and calls the function insert() after it  matched the input, passing along a pair of strings &#8220;a&#8221; and &#8220;b&#8221;.<\/p>\n<p>It stops compiling if the semantic action is moved into the  assignment rule:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nassignment = (identifier &gt;&gt; '=' &gt;&gt; identifier &gt;&gt; ';')[&amp;insert];\r\nqi::parse(input.begin(), input.end(), assignment);\r\n<\/pre>\n<p>On the other hand, it still compiles and runs if the code is written  as:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nassignment = (identifier &gt;&gt; '=' &gt;&gt; identifier &gt;&gt; ';');\r\nstd::pair&lt;std::string, std::string&gt; pp;\r\nqi::parse(input.begin(), input.end(), assignment, pp);\r\n<\/pre>\n<p>What (the hell) is going on? Any help with an explanation would be  appreciated!<\/p>\n<h5>The Attribute Rules<\/h5>\n<p>Attribute handling in Spirit is governed by two seemingly  contradictory set of rules: attribute <em>propagation<\/em> rules and  attribute <em>compatibility<\/em> rules.<\/p>\n<p>The attribute <em>propagation<\/em> rules are well formalized and are  described in the documentation for each of the components. They are  applied from the &#8216;bottom up&#8217;, describing how the exposed attributes of  the components compose to form the overall attribute of a <em>Spirit<\/em> expression. For instance, for sequences we find the general rule to be  (see the docs <a href=\"http:\/\/www.boost.org\/doc\/libs\/1_45_0\/libs\/spirit\/doc\/html\/spirit\/qi\/reference\/operator\/sequence.html#spirit.qi.reference.operator.sequence.attributes\">here<\/a>):<\/p>\n<blockquote><p>a: A, b: B &#8211;&gt; (a &gt;&gt; b): tuple&lt;A, B&gt;<\/p><\/blockquote>\n<p>which reads as:<\/p>\n<blockquote><p>Given the parser component a exposes an attribute of type  A and the parser component b exposes an attribute of type B, then the  sequence of parser components a &gt;&gt; b will expose an attribute of  type tuple&lt;A, B&gt;.<\/p><\/blockquote>\n<p>In terms of attribute <em>propagation<\/em>, tuple stands for  fusion::vector.<\/p>\n<p>At the same time, the use of &#8216;tuple&#8217; above already hints at the  attribute <em>compatibility<\/em> rules. It means, that a sequence will be  able to fill any attribute as long as it is a Fusion sequence of any  type (for instance std::pair) holding two elements of type A and B.<\/p>\n<p>The attribute <em>compatibility<\/em> rules are applied from the &#8216;top  downwards&#8217;. They prescribe whether a user supplied attribute can be  directly used by the component it is passed to. <em>Spirit<\/em> requires a  given attribute type to conform to a set of concepts in order for this  type to be compatible with a component. These concepts are specific to  every parser or generator component. Unfortunately the current  documentation does not contain a full and explicit description of those  conceptual requirements, which is something we definitely need to  improve on in the future.<\/p>\n<p><em>Spirit<\/em> enforces these concepts (i.e. the compatibility rules)  by wrapping all attribute handling into special templates, called  customization points. All default implementations of the customization  points (CPs) generally expose a functionality, which is aligned with  what a user would intuitively expect. The tricky part is that you can  provide specializations for all CPs, therefore bending the rules for  your types. However, here are the three main rules of attribute  compatibility in <em>Spirit<\/em>:<\/p>\n<ul>\n<li>Normal C++ convertibility applies in any case: two types A and B are compatible if there exists a default conversion from A to B,<\/li>\n<li>STL containers are compatible amongst each other as long as their  value_type is compatible<\/li>\n<li>All Fusion sequences are compatible if they have the same length and  all of their elements are pairwise compatible.<\/li>\n<\/ul>\n<p>The main exception are sequences, which in addition to their exposed  attribute being a Fusion sequence, may be compatible with STL containers  if all elements of that sequence are either compatible with the given  container or its value_type.<\/p>\n<p>The attribute <em>compatibility<\/em> rules generally have precedence  over the attribute <em>propagation<\/em> rules and they are applied whenever  possible, fully disabling attribute <em>propagation<\/em>. On the other  hand, attribute <em>propagation<\/em> rules have preference whenever the  compatibility rules cannot be applied. This happens:<\/p>\n<ul>\n<li>If no (explicit) user supplied attribute is available,<\/li>\n<li>Semantic actions are involved,<\/li>\n<li>Non-terminals are involved, i.e. for constructs using rule&lt;&gt;,  grammar&lt;&gt;, and subrule&lt;&gt;,<\/li>\n<li>The user has requested an explicit attribute conversion using  attr_cast&lt;&gt;<\/li>\n<\/ul>\n<p>Each of those cases influences the attribute handling in slightly  different ways. We describe the first three below, leaving out  attr_cast&lt;&gt; as it exposes semantics very similar to non-terminals.<\/p>\n<p><em>Edit<\/em>: some people have been asking what the phrase &#8216;Semantic actions are involved&#8217; means. It means that attribute compatibility is disabled for a specific rule if the rule&#8217;s right hand side (directly) has any semantic actions attached. Semantic actions attached to rules invoked from the right hand side are irrelevant for this specific rule:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\n\/\/ compatibility disabled for 'a'\r\nrule&lt;Iterator, int()&gt; a = int_[f];\r\n\/\/ compatibility enforced for 'b'\r\nrule&lt;Iterator, int()&gt; b; b %= int_[f];\r\n\/\/ compatibility enabled for 'r'\r\nrule&lt;Iterator, std::vector&lt;int&gt;()&gt; r = a &gt;&gt; b; \r\n<\/pre>\n<h5>Missing Explicit Attribute<\/h5>\n<p>If no user supplied attribute is given, the user either does not want  to handle attributes at all (for instance, using a parser to verify any  given input only), or he\/she uses semantic actions to perform operations  related to attributes. We generally assume that as soon as semantic actions are attached to an expression, the user takes full responsibility for attribute handling. Here is a simple example:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nusing boost::phoenix::ref;\r\nint i = 0;\r\nstd::string input(&quot;123&quot;);\r\nqi::parse(input.begin(), input.end(), qi::int_[ref(i) = qi::_1]);\r\n<\/pre>\n<p>In this case no attribute is passed to (forced upon) the parser,  therefore the placeholder qi::_1 refers to the attribute exposed  (propagated) from the parser the semantic action is attached to (the  qi::int_): qi::_1 refers to an int. This is caused by the action  component, which <em>always<\/em> makes sure that the attribute type passed  to the semantic action is the same as the exposed (propagated) attribute of the  component it is attached to.<\/p>\n<h5>Non-Terminals<\/h5>\n<p>Non-terminals expose a well defined attribute type \u2013 it is specified  as one of the template parameters. For instance, a <em>Qi<\/em> rule  exposing a std::string has to be defined as:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nqi::rule&lt;iterator, std::string()&gt; identifier;\r\nidentifier = qi::alpha &gt;&gt; *qi::alnum;\r\n<\/pre>\n<p>Any right hand side parser for this rule will be invoked with an  attribute of type std::string (if no semantic actions are attached to  any of its parts). This mainly implies that the rule&#8217;s attribute type  has to be <em>compatible<\/em> with the right hand side component (the rule  passes an instance of the given attribute type to its right hand side).<\/p>\n<p>At the same time, if the right hand side has a semantic action  attached, no attribute is passed to it:<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nqi::rule&lt;iterator&gt; identifier;\r\nidentifier = (qi::alpha &gt;&gt; *qi::alnum)[&amp;store_ident];\r\n<\/pre>\n<p>This forces the action component to synthesize a new attribute  instance. In this case, in accordance with the known attribute  propagation rules, the type of this synthesized attribute is  fusion::vector2&lt;char, std::vector&lt;char&gt; &gt;, which is passed along to the semantic action.  Obviously, the function store_ident has to accept a parameter of this  type as well.<\/p>\n<p>Sometimes it is necessary to combine both, semantic actions and  attribute compatibility. <em>Spirit<\/em> allows to enforce this by using the  operator%=():<\/p>\n<pre class=\"brush: cpp; title: ; notranslate\" title=\"\">\r\nqi::rule&lt;iterator, std::string()&gt; identifier;\r\nidentifier %=\r\n    qi::string(&quot;ident:&quot;) &gt;&gt; (qi::alpha &gt;&gt; *qi::alnum)[&amp;store_ident];\r\n<\/pre>\n<p>Here the rule&#8217;s attribute will be passed to the first component in  the right hand side&#8217;s sequence (the qi::string(&#8220;ident:&#8221;)), but the  function store_ident will still be called with the fusion::vector2 as  described above.<\/p>\n<h5>Back to the Question at Hand<\/h5>\n<p>Let&#8217;s apply the newly gained knowledge to the examples given in the  original question.<\/p>\n<p>The first example above compiles fine because the semantic action is  attached to the rule assignment, which exposes the std::pair&lt;&gt; as  its attribute. This causes the semantic action to be invoked with that  very same std::pair&lt;&gt;. At the same time the rule&#8217;s  std::pair&lt;&gt; attribute is compatible with the sequence at its right  hand side (both types are Fusion sequences of the same length and hold  the same element types).<\/p>\n<p>The second example does not compile, because the semantic action is  attached to the sequence. According to the attribute propagation rules  for sequences, its attribute type will be a fusion::vector2, which is  exactly the type of the attribute passed to the function insert. We  cannot (default-) convert the Fusion vector2&lt;&gt; to a  std::pair&lt;&gt;, which causes the compilation error.<\/p>\n<p>The third example above does compile. We can apply the <em>compatibility<\/em> rules for all components involved. The std::pair&lt;&gt; is passed from  the very top is compatible with the rule assignment and with the rule&#8217;s  right hand side. The attribute is handed down to the single components,  splitting of the original std::pair&lt;&gt;, passing one of the pair&#8217;s std::strings at  a time to each of the elements of the sequence.<\/p>\n<h5>Conclusion<\/h5>\n<p>Given all the complexity of attribute <em>propagation<\/em> and attribute <em>compatibility<\/em> we sometimes wish to have chosen a simpler path. At the same time  anything simpler than what we have would be a lot less powerful and less  flexible. We have had a lot of discussions whether to abandon the  attribute compatibility rules in future versions of <em>Spirit<\/em> all together.  But we always come back to the understanding that overall the current scheme is a good thing. It took us a while to remove the major problems  and to reach a sufficient level of consistency. However we believe,  that <em>Spirit<\/em> provides you with a powerful and tunable way of  handling attributes, which is one of the biggest innovations introduced  by this library.<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li><a href=\"#\" class=\"sharing-anchor sd-button share-more\"><span>Share<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><div class=\"sharing-hidden\"><div class=\"inner\" style=\"display: none;\"><ul><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-1334\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-1334\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-pinterest\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-pinterest-1334\" class=\"share-pinterest sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=pinterest\" target=\"_blank\" title=\"Click to share on Pinterest\" ><span>Pinterest<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-1334\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-reddit\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-reddit sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=reddit\" target=\"_blank\" title=\"Click to share on Reddit\" ><span>Reddit<\/span><\/a><\/li><li class=\"share-tumblr\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-tumblr sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=tumblr\" target=\"_blank\" title=\"Click to share on Tumblr\" ><span>Tumblr<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Narinder Claire asked a seemingly innocent question on the Spirit mailing list the other day. After starting to write an answer I realized that this question is not innocent at all as it touches the very fabric of Spirit: the rules of attribute handling. Many people have a hard time to properly understand what is [&hellip;]<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li><a href=\"#\" class=\"sharing-anchor sd-button share-more\"><span>Share<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><div class=\"sharing-hidden\"><div class=\"inner\" style=\"display: none;\"><ul><li class=\"share-facebook\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-facebook-1334\" class=\"share-facebook sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\" ><span>Facebook<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-twitter-1334\" class=\"share-twitter sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\" ><span>Twitter<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-pinterest\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-pinterest-1334\" class=\"share-pinterest sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=pinterest\" target=\"_blank\" title=\"Click to share on Pinterest\" ><span>Pinterest<\/span><\/a><\/li><li class=\"share-linkedin\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"sharing-linkedin-1334\" class=\"share-linkedin sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\" ><span>LinkedIn<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-reddit\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-reddit sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=reddit\" target=\"_blank\" title=\"Click to share on Reddit\" ><span>Reddit<\/span><\/a><\/li><li class=\"share-tumblr\"><a rel=\"nofollow noopener noreferrer\" data-shared=\"\" class=\"share-tumblr sd-button share-icon\" href=\"http:\/\/boost-spirit.com\/home\/articles\/attribute_handling\/attribute-propagation-and-attribute-compatibility\/?share=tumblr\" target=\"_blank\" title=\"Click to share on Tumblr\" ><span>Tumblr<\/span><\/a><\/li><li class=\"share-end\"><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div><\/div><\/div>","protected":false},"author":3,"featured_media":0,"parent":422,"menu_order":4,"comment_status":"open","ping_status":"open","template":"article-page.php","meta":{"_s2mail":"yes","spay_email":""},"jetpack_shortlink":"https:\/\/wp.me\/PIHdZ-lw","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/1334"}],"collection":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/comments?post=1334"}],"version-history":[{"count":15,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/1334\/revisions"}],"predecessor-version":[{"id":1495,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/1334\/revisions\/1495"}],"up":[{"embeddable":true,"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/pages\/422"}],"wp:attachment":[{"href":"http:\/\/boost-spirit.com\/home\/wp-json\/wp\/v2\/media?parent=1334"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}