Nov 19

Debugging

By Joel de Guzman Add comments

Preliminary documentation for Spirit2 debugging support.

Requirements:

  1. It should provide info on rule attributes and locals.
  2. It should be easy to set a breakpoint for conditional debugging in the IDE.
  3. The user should be able to easily supply her own debug handler.
  4. The output should be valid xml (snippet) that can be read by any xml editor.

One thing that’s nice about having an xml format for debugging is that you can use an xml editor or viewer to inspect the results. A good debugging strategy is to collect the debug result into a text editor and inspect it with an XML editor. Some (most?) editors have display filter capabilities to make it easy to spot the piece of XML you are looking for.

Here’s how to add debugging to a rule in Spirit 2.1:

debug(my_rule)

Yep, that was easy. That one uses Spirit 2.1′s simple_trace class which provides simple tracing capabilities (similar to Classic’s). Be aware that:

  1. Your attributes and local variables need to have a streaming operator defined, otherwise you’ll be getting a compiler error.
  2. That your rule is named. Check the docs on how to do this. Otherwise, you will get a printout with ‘unamed-rule’. If you want to have the rule named and enable debugging at the same time, you can use the convenience macro: BOOST_SPIRIT_DEBUG_NODE(my_rule).

The example calc4_debug.cpp highlights the debugger. Here’s a sample session:

1+1
<expression>
   <try>1+1</try>
   <term>
     <try>1+1</try>
     <factor>
       <try>1+1</try>
       <success>+1</success>
       <attributes>(1)</attributes>
     </factor>
     <success>+1</success>
     <attributes>(1)</attributes>
   </term>
   <term>
     <try>1</try>
     <factor>
       <try>1</try>
       <success></success>
       <attributes>(1)</attributes>
     </factor>
     <success></success>
     <attributes>(1)</attributes>
   </term>
   <success></success>
   <attributes>(2)</attributes>
</expression>
-------------------------
Parsing succeeded
result = 2
-------------------------

Things to note:

  • The printout is more xml-ish than before. There is no wierd <#tag>to signal failure. You can use an xml editor to analyze the result.
  • The attributes are printed. It is a tuple printed inside the (). What you see is the synthesized attribute of the rule. If there are inherited attributes, they are also printed. If there are local variables, those are also printed (a line after “Attributes”). The format is:
    <attributes>(Synth, Inh1, Inh2 ... InhN)</attributes>
    <locals>(Loc1, Loc2 ... LocN)</locals>
    
  • The markup
    <try>...</try> and <success>...</success>

    show some of the inputs prior and after parsing. As before, the number of chars printed is controlled by: BOOST_SPIRIT_DEBUG_PRINT_SOME

  • As before the output is controlled by: BOOST_SPIRIT_DEBUG_OUT
  • The number of indents os controlled by: BOOST_SPIRIT_DEBUG_INDENT

Here’s a sample with syntax error:

1+a
<expression>
   <try>1+a</try>
   <term>
     <try>1+a</try>
     <factor>
       <try>1+a</try>
       <success>+a</success>
       <attributes>(1)</attributes>
     </factor>
     <success>+a</success>
     <attributes>(1)</attributes>
   </term>
   <term>
     <try>a</try>
     <factor>
       <try>a</try>
       <fail/>
     </factor>
     <fail/>
   </term>
Error! Expecting <term> here: "a"
   <fail/>
</expression>
-------------------------
Parsing failed
-------------------------

Notes:

  1. Error! Expecting <term> here: “a” was printed by the error_handler (same one in calc4.cpp). It shows in context where the exception was caught
  2. <fail/> signals parse failure.

As mentioned, the calc4_debug.cpp example utilizes the Spirit supplied simple_trace class. It is a simple function object. You can set a breakpoint in its operator(). The class is very simple:

struct simple_trace
{
    void print_indent(int n) const
    {
        n *= BOOST_SPIRIT_DEBUG_INDENT;
        for (int i = 0; i != n; ++i)
            BOOST_SPIRIT_DEBUG_OUT << ' ';
    }

    template <typename Iterator>
    void print_some(
        char const* tag
      , int indent
      , Iterator first, Iterator const& last) const
    {
        print_indent(indent);
        BOOST_SPIRIT_DEBUG_OUT << '<' << tag << '>';
        int const n = BOOST_SPIRIT_DEBUG_PRINT_SOME;
        for (int i = 0; first != last && i != n; ++i)
            BOOST_SPIRIT_DEBUG_OUT << *first++;
        BOOST_SPIRIT_DEBUG_OUT << "</" << tag << '>' << std::endl;
    }

    template <typename Iterator, typename Context, typename State>
    void operator()(
        Iterator const& first
      , Iterator const& last
      , Context const& context
      , State state
      , std::string const& rule_name) const
    {
        int static indent = 0;

        switch (state)
        {
            case pre_parse:
                print_indent(indent++);
                BOOST_SPIRIT_DEBUG_OUT
                    << '<' << rule_name << '>'
                    << std::endl;
                print_some("try", indent, first, last);
                break;
            case successful_parse:
                print_some("success", indent, first, last);
                print_indent(indent);
                BOOST_SPIRIT_DEBUG_OUT
                    << "<attributes>" <<
                        context.attributes << "</attributes>";
                if (!fusion::empty(context.locals))
                    BOOST_SPIRIT_DEBUG_OUT << "<locals>"
                        << context.locals << "</locals>";
                BOOST_SPIRIT_DEBUG_OUT << std::endl;
                print_indent(--indent);
                BOOST_SPIRIT_DEBUG_OUT << "</" << rule_name << '>'
                    << std::endl;
                break;
            case failed_parse:
                print_indent(indent);
                BOOST_SPIRIT_DEBUG_OUT
                    << "<fail/>" << std::endl;
                print_indent(--indent);
                BOOST_SPIRIT_DEBUG_OUT << "</" << rule_name << '>'
                    << std::endl;
                break;
        }
    }
};

State is:

enum debug_handler_state
{
    pre_parse
  , successful_parse
  , failed_parse
};

Context is the rule’s context where the attributes and locals reside. There is a public API in support/context.hpp. The actual class is quite simply a struct with 2 elements:

template <typename Attributes, typename Locals>
struct context
{
    /*...*/

    Attributes attributes;  // The attributes
    Locals locals;          // Local variables
};

Both are tuples.

simple_trace exemplifies a debug-handler. You can specify your own if needed:

debug(my_rule, my_handler);

The only requiremenmt is for the debug-handler to have an operator() with the signature:

template <typename Iterator, typename Context, typename State>
void operator()(
    Iterator const& first
  , Iterator const& last
  , Context const& context
  , State state
  , std::string const& rule_name) const;

Ok, there you go… As usual, comments and suggestionsare very welcome. Have fun debugging!

9 Responses to “Debugging”

  1. udpn says:

    This one was very useful. Why not include it into the manual?

  2. Joel de Guzman says:

    Most(*) of what you see here will eventually make it into the manual.

    (*) Except those that will be superseded or become obsolete.

  3. Rob says:

    Here are a number of questions which should be addressed in the documentation. You might update this post to address them in the interim.

    Must one add “debug(my_rule)” to each rule or just the start rule? According to the referenced example, it would seem that each rule must be marked separately, but you don’t say so.

    Where should that statement appear? In the grammar’s constructor? The example would seem to indicate as much, but you should be explicit.

    Must one write the equivalent of simple_trace to get output or is that just illustrating what one can do should something special be wanted? I was seeing some compilation errors initially that referenced simple_trace.hpp without my having done anything knowingly to reference it, but I got no debugging output. The example code makes it look like I should just get output once I mark rules with debug(). What I discovered was that the debug() lines must follow the rules’ initialization, so they should be last in the constructor body (assuming that is the intended placement). It may be that other constructs can follow the debug() statements, but you should make clear what they must follow.

    In Spirit Classic, one had to define BOOST_SPIRIT_DEBUG to enable debugging support. That appears to not be the case in Spirit v2, but then how is debugging disabled? Do we just conditionally compile the debug() lines?

  4. Hartmut Kaiser says:

    Rob, all your comments are valid. “debug(my_rule)” needs to be called for each rule you would like to see debug output generated for. It has to be called after the rule was initialized. There is no need to write your own simple_trace as Spirit provides that for you. It is possible to supply your own formatting, though.

    BOOST_SPIRIT_DEBUG needs to be defined only if you’re using BOOST_SPIRIT_DEBUG_NODE(my_rule) instead of debug(my_rule), allowing to control whether debug output will be generated with a single PP constant.

  5. Rob says:

    simple_trace doesn’t work well in the face of exceptions. The closing delimiters are not written, the indentation is not reset, and there’s no indication of an exception being thrown.

    • Hartmut Kaiser says:

      Hey, it’s called simple_trace :-P

      But seriously, thanks for reporting these problems, apparently nobody thought about this use case in the first place. Would you be able to provide a patch?

  6. Rob says:

    Another omission from this guidance: The rules must be named before calling debug().

    What’s more, in my testing with Boost 1.44, giving the rules a name in the initializer list does not name them for debugging purposes. Thus, one must do either:


      rule.name("rule");
      debug(rule);

    or:


    #define BOOST_SPIRIT_DEBUG
    struct grammar : boost::spirit::qi::grammar...
    {
       grammar()
          : grammar::base_type(rule)
       {
          BOOST_SPIRIT_DEBUG_NODE(rule);

    Note that BOOST_SPIRIT_DEBUG must be defined before including any Spirit headers.

  7. dariomt says:

    I think it’s worth mentioning that the debug(rule) *MUST* happen *AFTER* the rule has been defined.

    I’ve spent a couple of hours debugging Spirit code trying to figure out why this code didn’t show any debugging information:
    debug(r);
    r = /* some rule definition */;

    until I found that this code did work:
    r = /* some rule definition */;
    debug(r);

  8. One clarification to Rob’s last suggestion. The macro should be placed after rule assignment, not before:

    grammar() : grammar::base_type(rule)
    {
    rule = …;

    BOOST_SPIRIT_DEBUG_NODE(rule);
    }

Leave a Reply

To use RetinaPost you must register at http://www.RetinaPost.com/register