Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Re: [SQLWindowing] Windowing function output path syntax (#26)


+
neelesh gadhia 2013-02-22, 22:05
+
Butani, Harish 2013-02-22, 22:44
+
neelesh gadhia 2013-02-22, 22:55
+
Butani, Harish 2013-02-23, 06:40
Copy link to this message
-
Re: [SQLWindowing] Windowing function output path syntax (#26)
Hi Harish,

I did download the latest code from ptf-windowing branch on Feb 19 1:14 am PST. See attached screen shot of the location I used to download.

I will try to download again and build it and update you with failed task log if I am able to reproduce the issue.

thanks,
Neelesh

________________________________
 From: "Butani, Harish" <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; neelesh gadhia <[EMAIL PROTECTED]>; Ashutosh Chauhan <[EMAIL PROTECTED]>
Sent: Friday, February 22, 2013 10:40 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
 

 The plan dump indicates that you are not using the latest code from the ptf-windowing branch. So first of all, if you can please try with the latest code.
Otherwise can you post the log file of the failed task; and also tell us which version of the code you are using. 

Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]>
Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, neelesh gadhia <[EMAIL PROTECTED]>
Date: Friday, February 22, 2013 2:55 PM
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Thanks Harish for your quick response.

I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?

hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
    > from t_enc
    > distribute by mid
    > sort by mid, tdate;

1.TS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

2.RS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

3.EX :
RowResolver::
    columns:[t_enc._col0, t_enc._col1, t_enc._col2]
    Aliases:[
        t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
    ]
    columns mapped to expressions:[
    ]

4.PTF :
RowResolver::
    columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
    Aliases:[
        :[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
        t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
    ]
    columns mapped to expressions:[
        (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
    ]

insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3

1.TS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

2.RS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

3.EX :
RowResolver::
    columns:[t_enc._col0, t_enc._col1, t_enc._col2]
    Aliases:[
        t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
    ]
    columns mapped to expressions:[
    ]

4.PTF :
RowResolver::
    columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
    Aliases:[
        :[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
        t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
    ]
    columns mapped to expressions:[
        (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
    ]

5.SEL :
RowResolver::
    columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
    Aliases:[
        <null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
    ]
    columns mapped to expressions:[
    ]

6.FS :
RowResolver::
    columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
    Aliases:[
        <null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
    ]
    columns mapped to expressions:[
    ]

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job  -kill job_201302221435_0001
Hadoop job informatio
+
neelesh gadhia 2013-02-25, 04:26
+
Butani, Harish 2013-02-25, 05:22
+
neelesh gadhia 2013-02-25, 06:01
+
Butani, Harish 2013-02-25, 17:53
+
neelesh gadhia 2013-02-25, 18:55
+
Butani, Harish 2013-02-26, 03:57