Обсуждение: Why my manualy constructed raw parser tree produce failed to execute?

Hi all,

I have try to understand how parser work. So far the raw parser in
PostgreSQL will produce that called "raw parser tree" and the "raw
parser tree" will be passed to "analyzer", right?

I have modified PostgreSQL, so the program wont call function
"raw_parser" -a function container that make by yacc and lex-. The part
that i try to modified was at "src/backend/tcop/postgres.c" function
"pg_parse_query". I have using GDB to see list and node structure (a
parse tree structure that made using yacc) of my working query.

I have constructed same list and node structure as my working query,
using manual code (lit_make1, lappend, makeNode, etc). GDB pprint show
that my manualy constructed list are identically with my working query,
at least when i try to compare it by my eye 1 on 1. But when my manualy
constructed query tree is execute, it produce "error: unrecognized node
type".

Now to the question, why my manualy constructed list was failed to
execute? I was pretty sure that my list node was identical with yacc.

Is there something that i miss when i consctructed my list (perhaps some
list structure part that not printed by GDB)? By the way in GBD i using
"call pprint(node/list name)" to display my list.

Thank You.
-- 
Mohammad Heykal Abdillah <heykal.abdillah@gmail.com>

On Thu, May 27, 2010 at 1:58 PM, Mohammad Heykal Abdillah
<heykal.abdillah@gmail.com> wrote:
> Now to the question, why my manualy constructed list was failed to
> execute? I was pretty sure that my list node was identical with yacc.

Because you have a bug in your code.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

On Kam, 2010-05-27 at 15:02 -0400, Robert Haas wrote: 
> On Thu, May 27, 2010 at 1:58 PM, Mohammad Heykal Abdillah
> <heykal.abdillah@gmail.com> wrote:
> > Now to the question, why my manualy constructed list was failed to
> > execute? I was pretty sure that my list node was identical with yacc.
> 
> Because you have a bug in your code.
> 
Yes, that i know.

Anyway, this is my manualy generate list node i hope you can point me
where it went wrong :
In function "pg_parse_query(const char *query_string)"
 List    *my_parsetree_list; my_parsetree_list = NIL;
 ColumnRef *o = makeNode(ColumnRef);                 o->type = T_ColumnRef;                 o->fields =
list_make1(makeString("*"));                o->location = 16;
 
 ResTarget *m = makeNode(ResTarget);                   m->type = T_ResTarget;                   m->name = NIL;
        m->indirection = NIL;                   m->val = o;                   m->location = 16;
 
 RangeVar *p = makeNode(RangeVar);                 p->schemaname = NIL;               p->relname = "customer";
p->inhOpt= 2;      p->istemp = false ;     p->alias = NIL;
 
 SelectStmt *n         = makeNode(SelectStmt);     n->type           = T_SelectStmt;     n->distinctClause =
list_make1(NIL);;    n->intoClause     = NULL;       n->targetList     = list_make1(m);     n->fromClause     =
list_make1(p);      n->whereClause    = NULL;       n->groupClause    = NIL;      n->havingClause   = NULL;
n->valuesLists   = NIL;      n->sortClause     = NIL;      n->limitOffset    = NULL;       n->limitCount     = NULL;
     n->lockingClause  = NIL;      n->op             = 0;     n->all            = false;     n->larg           = NULL;
  n->rarg           = NULL;
 
  my_parsetree_list = list_make1(n);

raw_parsetree_list = my_parsetree_list;
if (log_parser_stats)
ShowUsage("PARSER STATISTICS"); ...
... and rest its same with original code.

Thank You
-- 
Mohammad Heykal Abdillah <heykal.abdillah@gmail.com>

> > Now to the question, why my manualy constructed list was failed to
> > execute? I was pretty sure that my list node was identical with yacc.
> 
> Because you have a bug in your code.

You can debug your code by comparing your hand made tree with the
original tree by using equal(). Search #ifdef COPY_PARSE_PLAN_TREES in
postgres.c to see how to use it. Good luck.
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese: http://www.sraoss.co.jp

On Thu, May 27, 2010 at 6:56 PM, Mohammad Heykal Abdillah
<heykal.abdillah@gmail.com> wrote:
> On Kam, 2010-05-27 at 15:02 -0400, Robert Haas wrote:
>> On Thu, May 27, 2010 at 1:58 PM, Mohammad Heykal Abdillah
>> <heykal.abdillah@gmail.com> wrote:
>> > Now to the question, why my manualy constructed list was failed to
>> > execute? I was pretty sure that my list node was identical with yacc.
>>
>> Because you have a bug in your code.
>>
> Yes, that i know.
>
> Anyway, this is my manualy generate list node i hope you can point me
> where it went wrong :
> In function "pg_parse_query(const char *query_string)"
>
>  List    *my_parsetree_list;
>  my_parsetree_list = NIL;
>
>  ColumnRef *o = makeNode(ColumnRef);
>                  o->type = T_ColumnRef;
>                  o->fields = list_make1(makeString("*"));
>                  o->location = 16;
>
>  ResTarget *m = makeNode(ResTarget);
>                    m->type = T_ResTarget;
>                    m->name = NIL;
>                    m->indirection = NIL;
>                    m->val = o;
>                    m->location = 16;
>
>  RangeVar *p = makeNode(RangeVar);
>                p->schemaname = NIL;
>                p->relname = "customer";
>      p->inhOpt = 2;
>      p->istemp = false ;
>      p->alias = NIL;
>
>  SelectStmt *n         = makeNode(SelectStmt);
>      n->type           = T_SelectStmt;
>      n->distinctClause = list_make1(NIL);;
>      n->intoClause     = NULL;
>      n->targetList     = list_make1(m);
>      n->fromClause     = list_make1(p);
>      n->whereClause    = NULL;
>      n->groupClause    = NIL;
>      n->havingClause   = NULL;
>      n->valuesLists    = NIL;
>      n->sortClause     = NIL;
>      n->limitOffset    = NULL;
>      n->limitCount     = NULL;
>      n->lockingClause  = NIL;
>      n->op             = 0;
>      n->all            = false;
>      n->larg           = NULL;
>      n->rarg           = NULL;
>
>   my_parsetree_list = list_make1(n);
>
> raw_parsetree_list = my_parsetree_list;
> if (log_parser_stats)
> ShowUsage("PARSER STATISTICS"); ...
> ... and rest its same with original code.

In order to debug this, I (or someone else) would need an actual patch
that could be applied to the source tree with a specific series of
steps that would reproduce the bug.  But I'm not really that
interested in that myself, since this isn't something that is going to
be integrated into the Postgres source code.  If you want to try to
debug it yourself, I suggest using equal() [as Tatsuo-san already
mentioned] or else stepping through the code line by line with a
debugger until you find where it's going wrong, or else adding some
printf() statements to print out key variables in the functions that
are being called and try to find out where the difference is between
the original code and your modified code.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company

On Jum, 2010-05-28 at 08:55 +0900, Tatsuo Ishii wrote: 
> > > Now to the question, why my manualy constructed list was failed to
> > > execute? I was pretty sure that my list node was identical with yacc.
> > 
> > Because you have a bug in your code.
> 
> You can debug your code by comparing your hand made tree with the
> original tree by using equal(). Search #ifdef COPY_PARSE_PLAN_TREES in
> postgres.c to see how to use it. Good luck.

Thank you Mr. Tatsuo and Mr. Robert for the hint, i think i'll have to
debug my code more thoroughly but at least now i have some hint :)

Once again thank you.

> --
> Tatsuo Ishii
> SRA OSS, Inc. Japan
> English: http://www.sraoss.co.jp/index_en.php
> Japanese: http://www.sraoss.co.jp


-- 
Mohammad Heykal Abdillah <heykal.abdillah@gmail.com>