Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Re: [SQLWindowing] Windowing function output path syntax (#26)


+
neelesh gadhia 2013-02-22, 22:05
+
Butani, Harish 2013-02-22, 22:44
+
neelesh gadhia 2013-02-22, 22:55
Copy link to this message
-
Re: [SQLWindowing] Windowing function output path syntax (#26)
Butani, Harish 2013-02-23, 06:40
The plan dump indicates that you are not using the latest code from the ptf-windowing branch. So first of all, if you can please try with the latest code.
Otherwise can you post the log file of the failed task; and also tell us which version of the code you are using.

Regards,
Harish.

From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Friday, February 22, 2013 2:55 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)

Thanks Harish for your quick response.

I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?

hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
    > from t_enc
    > distribute by mid
    > sort by mid, tdate;

1.TS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

2.RS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

3.EX :
RowResolver::
    columns:[t_enc._col0, t_enc._col1, t_enc._col2]
    Aliases:[
        t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
    ]
    columns mapped to expressions:[
    ]

4.PTF :
RowResolver::
    columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
    Aliases:[
        :[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
        t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
    ]
    columns mapped to expressions:[
        (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
    ]

insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3

1.TS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

2.RS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

3.EX :
RowResolver::
    columns:[t_enc._col0, t_enc._col1, t_enc._col2]
    Aliases:[
        t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
    ]
    columns mapped to expressions:[
    ]

4.PTF :
RowResolver::
    columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
    Aliases:[
        :[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
        t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
    ]
    columns mapped to expressions:[
        (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
    ]

5.SEL :
RowResolver::
    columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
    Aliases:[
        <null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
    ]
    columns mapped to expressions:[
    ]

6.FS :
RowResolver::
    columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
    Aliases:[
        <null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
    ]
    columns mapped to expressions:[
    ]

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job  -kill job_201302221435_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-22 14:52:43,467 Stage-1 map = 0%,  reduce = 0%
2013-02-22 14:53:05,568 Stage-1 map = 100%,  reduce = 100%
Ended Job = job_201302221435_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Examining task ID: task_2013022214
+
neelesh gadhia 2013-02-23, 07:00
+
neelesh gadhia 2013-02-25, 04:26
+
Butani, Harish 2013-02-25, 05:22
+
neelesh gadhia 2013-02-25, 06:01
+
Butani, Harish 2013-02-25, 17:53
+
neelesh gadhia 2013-02-25, 18:55
+
Butani, Harish 2013-02-26, 03:57