Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Re: [SQLWindowing] Windowing function output path syntax (#26)


+
neelesh gadhia 2013-02-22, 22:05
+
Butani, Harish 2013-02-22, 22:44
+
neelesh gadhia 2013-02-22, 22:55
+
Butani, Harish 2013-02-23, 06:40
Copy link to this message
-
Re: [SQLWindowing] Windowing function output path syntax (#26)
Hi Harish,

I did download the latest code from ptf-windowing branch on Feb 19 1:14 am PST. See attached screen shot of the location I used to download.

I will try to download again and build it and update you with failed task log if I am able to reproduce the issue.

thanks,
Neelesh

________________________________
 From: "Butani, Harish" <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; neelesh gadhia <[EMAIL PROTECTED]>; Ashutosh Chauhan <[EMAIL PROTECTED]>
Sent: Friday, February 22, 2013 10:40 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
 

 The plan dump indicates that you are not using the latest code from the ptf-windowing branch. So first of all, if you can please try with the latest code.
Otherwise can you post the log file of the failed task; and also tell us which version of the code you are using. 

Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]>
Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, neelesh gadhia <[EMAIL PROTECTED]>
Date: Friday, February 22, 2013 2:55 PM
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Thanks Harish for your quick response.

I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?

hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
    > from t_enc
    > distribute by mid
    > sort by mid, tdate;

1.TS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

2.RS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

3.EX :
RowResolver::
    columns:[t_enc._col0, t_enc._col1, t_enc._col2]
    Aliases:[
        t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
    ]
    columns mapped to expressions:[
    ]

4.PTF :
RowResolver::
    columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
    Aliases:[
        :[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
        t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
    ]
    columns mapped to expressions:[
        (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
    ]

insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3

1.TS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

2.RS :
RowResolver::
    columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
    Aliases:[
        t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
    ]
    columns mapped to expressions:[
    ]

3.EX :
RowResolver::
    columns:[t_enc._col0, t_enc._col1, t_enc._col2]
    Aliases:[
        t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
    ]
    columns mapped to expressions:[
    ]

4.PTF :
RowResolver::
    columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
    Aliases:[
        :[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
        t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
    ]
    columns mapped to expressions:[
        (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
    ]

5.SEL :
RowResolver::
    columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
    Aliases:[
        <null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
    ]
    columns mapped to expressions:[
    ]

6.FS :
RowResolver::
    columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
    Aliases:[
        <null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
    ]
    columns mapped to expressions:[
    ]

Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job  -kill job_201302221435_0001
Hadoop job informatio
+
neelesh gadhia 2013-02-25, 04:26
+
Butani, Harish 2013-02-25, 05:22
+
neelesh gadhia 2013-02-25, 06:01
+
Butani, Harish 2013-02-25, 17:53
+
neelesh gadhia 2013-02-25, 18:55
+
Butani, Harish 2013-02-26, 03:57
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB