{"id":836,"date":"2010-01-05T12:14:22","date_gmt":"2010-01-05T20:14:22","guid":{"rendered":"http:\/\/boost-spirit.com\/home\/?p=836"},"modified":"2010-01-22T08:15:04","modified_gmt":"2010-01-22T16:15:04","slug":"stream-based-parsing-made-easy","status":"publish","type":"post","link":"http:\/\/boost-spirit.com\/home\/2010\/01\/05\/stream-based-parsing-made-easy\/","title":{"rendered":"Stream-based Parsing Made Easy"},"content":{"rendered":"

We got many questions how to parse data which has to be accessed using a standard input stream (usually a std::istream<\/span>) without having to read all of the data into memory first.<\/p>\n

The standard way of iterating over a stream would be to wrap it into a std::istream_iterator<\/span>, but unfortunately this is not possible. Qi<\/em> requires iterators to be at least forward iterators<\/a>, while the std::istream_iterator<\/span> is an input iterator<\/a> only. To overcome this limitation Spirit V2.2 (the next version of Spirit, to be released with Boost V1.42) will implement a new iterator.<\/p>\n

<\/p>\n

\r\nnamespace boost { namespace spirit\r\n{\r\n\u00a0\u00a0\u00a0 \/\/ this is functionally equivalent to std::istream_iterator\r\n\u00a0\u00a0\u00a0 template <typename Char, typename Traits = std::char_traits<Char> >\r\n\u00a0\u00a0\u00a0 struct basic_istream_iterator;\r\n\r\n\u00a0\u00a0\u00a0 \/\/ predefine specialization for 'char'\r\n\u00a0\u00a0\u00a0 typedef basic_istream_iterator<char> istream_iterator;\r\n}}\r\n<\/pre>\n
This iterator is functionally equivalent to the std::istream_iterator<\/span> except that it is a forward iterator<\/a> you can utilize for your Qi parsing needs. Here is an example:<\/p>\n
\r\n#include <boost\/spirit\/include\/support_istream_iterator.hpp>\r\n\r\nnamespace spirit = boost::spirit;\r\n\r\n\/\/ open file, disable skipping of whitespace\r\nstd::ifstream in("some_data_file");\r\nin.unsetf(std::ios::skipws);\r\n\r\n\/\/ wrap istream into iterator\r\nspirit::istream_iterator begin(in);\r\nspirit::istream_iterator end;\r\n\r\n\/\/ use iterator to parse file data\r\nspirit::qi::parse(begin, end, <...your grammar here...>);\r\n<\/pre>\n
This iterator is implemented on top of the multi_pass<\/span> iterator framework, which is documented here<\/a>. For those curious enough to have a peek: you can access the new iterator from Boost<\/a> SVN here<\/a>. But in order to use it with Boost prior to V1.42 you will need to download the whole iterators directory<\/a> from SVN.<\/p>\n
Share this:<\/h3>