{"id":989,"date":"2010-02-24T05:32:08","date_gmt":"2010-02-24T13:32:08","guid":{"rendered":"http:\/\/boost-spirit.com\/home\/?p=989"},"modified":"2010-06-02T02:40:32","modified_gmt":"2010-06-02T09:40:32","slug":"parsing-skippers-and-skipping-parsers","status":"publish","type":"post","link":"http:\/\/boost-spirit.com\/home\/2010\/02\/24\/parsing-skippers-and-skipping-parsers\/","title":{"rendered":"Parsing Skippers and Skipping Parsers"},"content":{"rendered":"

Spirit<\/em> supports skipper based parsing since its very invention. So this is definitely not something new to Spirit V2. Nevertheless, the recent discussion on the Spirit mailing list<\/a> around the semantics of Qi’s<\/em> lexeme[]<\/span> directive shows the need for some clarification. Today I try to answer questions like: “What does it mean to use a skipper while parsing?”, or “When do I want to use a skipper and when not?”.<\/p>\n

<\/p>\n

While parsing some formatted data stream it is very often desirable to ignore some parts of the input. A common example would be the need to skip whitespace and comments while parsing some computer language. Certainly it is possible to explicitly account for the tokens to skip (such as the whitespace or the comments) while writing the grammar. But this can get very tedious as those tokens are valid to appear at any point in the input.<\/p>\n

For the sake of simplicity, let us assume we want to parse a simple key\/value expression: key=value<\/span>, where we want to allow for any number of space characters before, in between, or after the key<\/span> or the value<\/span>. A naive grammar matching the plain key\/value pair without whitespace skipping would look like (see Parsing a List of Key-Value Pairs Using Spirit.Qi<\/a> for more details):<\/p>\n

\r\npair  =  key >> '=' >> value;\r\nkey   =  qi::char_("a-zA-Z_") >> *qi::char_("a-zA-Z_0-9");\r\nvalue = +qi::char_("a-zA-Z_0-9");\r\n<\/pre>\n
If we want to explicitly accommodate the rule pair<\/span> to match any interspersed space characters we get:<\/p>\n
\r\npair  = *space >> key >> *space >> '=' *space >> value >> *space;\r\n<\/pre>\n
which, while it produces the desired result, is not only error prone, but additionally difficult to write, to understand, and to maintain. If we look closer we see, that the process of skipping the whitespace tokens is easily automated. It seems to be sufficient to insert a repeated invocation of the space<\/span> parser (or generally, any skip parser) in between the elements of the user defined parser expression sequences.<\/p>\n
In fact, that is exactly what Spirit<\/em> can do for you! The library invokes any supplied skip parser upon entry to the parse member function of any parser conforming to the PrimitiveParser<\/span><\/a> concept. The skip parser has to be supplied by calling a special API function: phrase_parse:<\/span><\/p>\n
\r\nnamespace qi = boost::spirit::qi;\r\ntypedef std::string::const_iterator iterator;\r\n\r\nqi::rule<iterator, qi::space_type> pair = key >> '=' >> value;\r\nqi::rule<iterator> key = qi::char_("a-zA-Z_") >> *qi::char_("a-zA-Z_0-9");\r\nqi::rule<iterator> value = +qi::char_("a-zA-Z_0-9");\r\n\r\nstd::string input(" key = value ");\r\niterator_type begin = input.begin();\r\niterator_type end = input.end();\r\nqi::phrase_parse(begin, end, pair, qi::space);\r\n<\/pre>\n
This code snippet illustrates several important things:<\/p>\n