neelesh gadhia
2013-02-22, 22:05
Butani, Harish
2013-02-22, 22:44
neelesh gadhia
2013-02-22, 22:55
Butani, Harish
2013-02-23, 06:40
neelesh gadhia
2013-02-23, 07:00
neelesh gadhia
2013-02-25, 04:26
Butani, Harish
2013-02-25, 05:22
neelesh gadhia
2013-02-25, 06:01
Butani, Harish
2013-02-25, 17:53
neelesh gadhia
2013-02-25, 18:55
Butani, Harish
2013-02-26, 03:57
-
Re: [SQLWindowing] Windowing function output path syntax (#26)neelesh gadhia 2013-02-22, 22:05
Hello,
I downloaded the source code from ptf-windowing branch and build the dist based on that.
Now when I try to make use of windowing function using the following ql, I get an error as shown below. Am I missing anything here? Please advise.
from <select mid, tdate, tamt from t_enc >
partition by mid
order by mid
with
sum(tamt) over rows between
unbounded preceding and current row as cum_amt
select mid,tdate,tamt,cum_amt;
hive> from <select mid, tdate, tamt from t_enc >
> partition by mid
> order by mid
> with
> sum(tamt) over rows between
> unbounded preceding and current row as cum_amt
> select mid,tdate,tamt,cum_amt;
NoViableAltException(258@[])
at org.apache.hadoop.hive.ql.parse.HiveParser.joinSource(HiveParser.java:32612)
at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:32498)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatement(HiveParser.java:26832)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:26716)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:981)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:687)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:444)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:416)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
FAILED: ParseException line 1:5 cannot recognize input near '<' 'select' 'mid' in join source
________________________________
From: hbutani <[EMAIL PROTECTED]>
To: hbutani/SQLWindowing <[EMAIL PROTECTED]>
Cc: ngadhia <[EMAIL PROTECTED]>
Sent: Sunday, February 17, 2013 4:50 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Hi,
We don't actively support this library anymore. This functionality is in the process of being folded into hive. You can see the latest code at https://github.com/apache/hive, the ptf-windowing branch. Also checkout the Jiras in Hive Jira: look for Jiras whose Component is PTF-Windowing.
regards,
Harish.
—
Reply to this email directly or view it on GitHub.
I downloaded the source code from ptf-windowing branch and build the dist based on that.
Now when I try to make use of windowing function using the following ql, I get an error as shown below. Am I missing anything here? Please advise.
from <select mid, tdate, tamt from t_enc >
partition by mid
order by mid
with
sum(tamt) over rows between
unbounded preceding and current row as cum_amt
select mid,tdate,tamt,cum_amt;
hive> from <select mid, tdate, tamt from t_enc >
> partition by mid
> order by mid
> with
> sum(tamt) over rows between
> unbounded preceding and current row as cum_amt
> select mid,tdate,tamt,cum_amt;
NoViableAltException(258@[])
at org.apache.hadoop.hive.ql.parse.HiveParser.joinSource(HiveParser.java:32612)
at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:32498)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatement(HiveParser.java:26832)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:26716)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:981)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:687)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:444)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:416)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
FAILED: ParseException line 1:5 cannot recognize input near '<' 'select' 'mid' in join source
________________________________
From: hbutani <[EMAIL PROTECTED]>
To: hbutani/SQLWindowing <[EMAIL PROTECTED]>
Cc: ngadhia <[EMAIL PROTECTED]>
Sent: Sunday, February 17, 2013 4:50 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Hi,
We don't actively support this library anymore. This functionality is in the process of being folded into hive. You can see the latest code at https://github.com/apache/hive, the ptf-windowing branch. Also checkout the Jiras in Hive Jira: look for Jiras whose Component is PTF-Windowing.
regards,
Harish.
—
Reply to this email directly or view it on GitHub.
+
neelesh gadhia 2013-02-22, 22:05
-
Re: [SQLWindowing] Windowing function output path syntax (#26)Butani, Harish 2013-02-22, 22:44
Hi Neelesh,
You are using the syntax from the SQLWindowing project; which was done on top of HQL.
Now the syntax is standard SQL; see ptf_general_queries.q for examples. Your e.g can be expressed as:
select sum(tamt) over (partition by mid order by mid rows between unbounded preceding and current row) as cum_amt,
mid,tdate,tamt,cum_amt
>From t_enc
Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Friday, February 22, 2013 2:05 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Hello,
I downloaded the source code from ptf-windowing branch and build the dist based on that.
Now when I try to make use of windowing function using the following ql, I get an error as shown below. Am I missing anything here? Please advise.
from <select mid, tdate, tamt from t_enc >
partition by mid
order by mid
with
sum(tamt) over rows between
unbounded preceding and current row as cum_amt
select mid,tdate,tamt,cum_amt;
hive> from <select mid, tdate, tamt from t_enc >
> partition by mid
> order by mid
> with
> sum(tamt) over rows between
> unbounded preceding and current row as cum_amt
> select mid,tdate,tamt,cum_amt;
NoViableAltException(258@[])
at org.apache.hadoop.hive.ql.parse.HiveParser.joinSource(HiveParser.java:32612)
at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:32498)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatement(HiveParser.java:26832)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:26716)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:981)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:687)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:444)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:416)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
FAILED: ParseException line 1:5 cannot recognize input near '<' 'select' 'mid' in join source
________________________________
From: hbutani <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: hbutani/SQLWindowing <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: ngadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Sunday, February 17, 2013 4:50 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Hi,
We don't actively support this library anymore. This functionality is in the process of being folded into hive. You can see the latest code at https://github.com/apache/hive, the ptf-windowing branch. Also checkout the Jiras in Hive Jira: look for Jiras whose Component is PTF-Windowing.
regards,
Harish.
—
Reply to this email directly or view it on GitHub<https://github.com/hbutani/SQLWindowing/issues/26#issuecomment-13701539>.
You are using the syntax from the SQLWindowing project; which was done on top of HQL.
Now the syntax is standard SQL; see ptf_general_queries.q for examples. Your e.g can be expressed as:
select sum(tamt) over (partition by mid order by mid rows between unbounded preceding and current row) as cum_amt,
mid,tdate,tamt,cum_amt
>From t_enc
Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Friday, February 22, 2013 2:05 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Hello,
I downloaded the source code from ptf-windowing branch and build the dist based on that.
Now when I try to make use of windowing function using the following ql, I get an error as shown below. Am I missing anything here? Please advise.
from <select mid, tdate, tamt from t_enc >
partition by mid
order by mid
with
sum(tamt) over rows between
unbounded preceding and current row as cum_amt
select mid,tdate,tamt,cum_amt;
hive> from <select mid, tdate, tamt from t_enc >
> partition by mid
> order by mid
> with
> sum(tamt) over rows between
> unbounded preceding and current row as cum_amt
> select mid,tdate,tamt,cum_amt;
NoViableAltException(258@[])
at org.apache.hadoop.hive.ql.parse.HiveParser.joinSource(HiveParser.java:32612)
at org.apache.hadoop.hive.ql.parse.HiveParser.fromClause(HiveParser.java:32498)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatement(HiveParser.java:26832)
at org.apache.hadoop.hive.ql.parse.HiveParser.queryStatementExpression(HiveParser.java:26716)
at org.apache.hadoop.hive.ql.parse.HiveParser.execStatement(HiveParser.java:981)
at org.apache.hadoop.hive.ql.parse.HiveParser.statement(HiveParser.java:687)
at org.apache.hadoop.hive.ql.parse.ParseDriver.parse(ParseDriver.java:444)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:416)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:335)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:898)
at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
FAILED: ParseException line 1:5 cannot recognize input near '<' 'select' 'mid' in join source
________________________________
From: hbutani <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: hbutani/SQLWindowing <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: ngadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Sunday, February 17, 2013 4:50 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Hi,
We don't actively support this library anymore. This functionality is in the process of being folded into hive. You can see the latest code at https://github.com/apache/hive, the ptf-windowing branch. Also checkout the Jiras in Hive Jira: look for Jiras whose Component is PTF-Windowing.
regards,
Harish.
—
Reply to this email directly or view it on GitHub<https://github.com/hbutani/SQLWindowing/issues/26#issuecomment-13701539>.
+
Butani, Harish 2013-02-22, 22:44
-
Re: [SQLWindowing] Windowing function output path syntax (#26)neelesh gadhia 2013-02-22, 22:55
Thanks Harish for your quick response.
I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
5.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
6.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302221435_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-22 14:52:43,467 Stage-1 map = 0%, reduce = 0%
2013-02-22 14:53:05,568 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201302221435_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Examining task ID: task_201302221435_0001_m_000002 (and more) from job job_201302221435_0001
Task with the most failures(4):
Task ID:
task_201302221435_0001_m_000000
URL:
http://localhost:50030/taskdetails.jsp?jobid=job_201302221435_0001&tipid=task_201302221435_0001_m_000000
Diagnostic Messages for this Task:
java.lang.RuntimeException: java.util.NoSuchElementException
at org.apache.hadoop.hive.ql.exec.Utilities.getMapRedWork(Utilities.java:228)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:381)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:374)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:540)
at org.apache.hadoop.mapred
I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
5.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
6.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302221435_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-22 14:52:43,467 Stage-1 map = 0%, reduce = 0%
2013-02-22 14:53:05,568 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201302221435_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Examining task ID: task_201302221435_0001_m_000002 (and more) from job job_201302221435_0001
Task with the most failures(4):
Task ID:
task_201302221435_0001_m_000000
URL:
http://localhost:50030/taskdetails.jsp?jobid=job_201302221435_0001&tipid=task_201302221435_0001_m_000000
Diagnostic Messages for this Task:
java.lang.RuntimeException: java.util.NoSuchElementException
at org.apache.hadoop.hive.ql.exec.Utilities.getMapRedWork(Utilities.java:228)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:255)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:381)
at org.apache.hadoop.hive.ql.io.HiveInputFormat.pushProjectionsAndFilters(HiveInputFormat.java:374)
at org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getRecordReader(CombineHiveInputFormat.java:540)
at org.apache.hadoop.mapred
+
neelesh gadhia 2013-02-22, 22:55
-
Re: [SQLWindowing] Windowing function output path syntax (#26)Butani, Harish 2013-02-23, 06:40
The plan dump indicates that you are not using the latest code from the ptf-windowing branch. So first of all, if you can please try with the latest code.
Otherwise can you post the log file of the failed task; and also tell us which version of the code you are using.
Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Friday, February 22, 2013 2:55 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Thanks Harish for your quick response.
I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
5.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
6.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302221435_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-22 14:52:43,467 Stage-1 map = 0%, reduce = 0%
2013-02-22 14:53:05,568 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201302221435_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Examining task ID: task_2013022214
Otherwise can you post the log file of the failed task; and also tell us which version of the code you are using.
Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Friday, February 22, 2013 2:55 PM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Thanks Harish for your quick response.
I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
5.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
6.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302221435_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-22 14:52:43,467 Stage-1 map = 0%, reduce = 0%
2013-02-22 14:53:05,568 Stage-1 map = 100%, reduce = 100%
Ended Job = job_201302221435_0001 with errors
Error during job, obtaining debugging information...
Job Tracking URL: http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Examining task ID: task_2013022214
+
Butani, Harish 2013-02-23, 06:40
-
Re: [SQLWindowing] Windowing function output path syntax (#26)neelesh gadhia 2013-02-23, 07:00
Hi Harish,
I did download the latest code from ptf-windowing branch on Feb 19 1:14 am PST. See attached screen shot of the location I used to download.
I will try to download again and build it and update you with failed task log if I am able to reproduce the issue.
thanks,
Neelesh
________________________________
From: "Butani, Harish" <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; neelesh gadhia <[EMAIL PROTECTED]>; Ashutosh Chauhan <[EMAIL PROTECTED]>
Sent: Friday, February 22, 2013 10:40 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
The plan dump indicates that you are not using the latest code from the ptf-windowing branch. So first of all, if you can please try with the latest code.
Otherwise can you post the log file of the failed task; and also tell us which version of the code you are using.
Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]>
Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, neelesh gadhia <[EMAIL PROTECTED]>
Date: Friday, February 22, 2013 2:55 PM
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Thanks Harish for your quick response.
I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
5.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
6.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302221435_0001
Hadoop job informatio
I did download the latest code from ptf-windowing branch on Feb 19 1:14 am PST. See attached screen shot of the location I used to download.
I will try to download again and build it and update you with failed task log if I am able to reproduce the issue.
thanks,
Neelesh
________________________________
From: "Butani, Harish" <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; neelesh gadhia <[EMAIL PROTECTED]>; Ashutosh Chauhan <[EMAIL PROTECTED]>
Sent: Friday, February 22, 2013 10:40 PM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
The plan dump indicates that you are not using the latest code from the ptf-windowing branch. So first of all, if you can please try with the latest code.
Otherwise can you post the log file of the failed task; and also tell us which version of the code you are using.
Regards,
Harish.
From: neelesh gadhia <[EMAIL PROTECTED]>
Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, neelesh gadhia <[EMAIL PROTECTED]>
Date: Friday, February 22, 2013 2:55 PM
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
Thanks Harish for your quick response.
I tried it with the new syntax using one of the example in ptf_general_queries.q and get the following error. Am I still doing something wrong here?
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
insclause-0:
Def ObjectInspector:[_col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
windowingtablefunction:
Def ObjectInspector:[com_sum, _col0, _col1, _col2]
SerDe:org.apache.hadoop.hive.serde2.lazybinary.LazyBinarySerDe
Evaluator Output ObjectInspector:[com_sum, _col0, _col1, _col2]
SelectList:_col0, _col1, _col2, _col3
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2
]
columns mapped to expressions:[
]
4.PTF :
RowResolver::
columns:[<null>._col0, t_enc._col1, t_enc._col2, t_enc._col3]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> _col0
t_enc:[mid -> _col1, tdate -> _col2, tamt -> _col3
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
5.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
6.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302221435_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302221435_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302221435_0001
Hadoop job informatio
+
neelesh gadhia 2013-02-23, 07:00
-
Re: [SQLWindowing] Windowing function output path syntax (#26)neelesh gadhia 2013-02-25, 04:26
Hi Harish,
I downloaded the latest code from https://github.com/apache/hive/tree/ptf-windowing ( ptf-windowing branch).
I ran the same sql and get a different error now. Please advise.
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242020_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242020_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242020_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-24 20:22:38,431 Stage-1 map = 0%, reduce = 0%
2013-02-24 20:22:40,451 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:22:48,505 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:22:52,530 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:23:00,566 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:23:04,586 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:23:12,625 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:23:16,645 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:2
I downloaded the latest code from https://github.com/apache/hive/tree/ptf-windowing ( ptf-windowing branch).
I ran the same sql and get a different error now. Please advise.
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242020_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242020_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242020_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-24 20:22:38,431 Stage-1 map = 0%, reduce = 0%
2013-02-24 20:22:40,451 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:22:48,505 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:22:52,530 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:23:00,566 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:23:04,586 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:23:12,625 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:23:16,645 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:2
+
neelesh gadhia 2013-02-25, 04:26
-
Re: [SQLWindowing] Windowing function output path syntax (#26)Butani, Harish 2013-02-25, 05:22
Can you try
Set hive.ptf.partition.persistence.memsize=something lower
The default is 64mb; will fix it so that the default is lower
On Feb 24, 2013, at 8:26 PM, "neelesh gadhia" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi Harish,
I downloaded the latest code from https://github.com/apache/hive/tree/ptf-windowing ( ptf-windowing branch).
I ran the same sql and get a different error now. Please advise.
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242020_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242020_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242020_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-24 20:22:38,431 Stage-1 map = 0%, reduce = 0%
2013-02-24 20:22:40,451 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:22:48,505 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:22:52,530 Stage-1 map = 100%, reduce = 0%
2013-02-24
Set hive.ptf.partition.persistence.memsize=something lower
The default is 64mb; will fix it so that the default is lower
On Feb 24, 2013, at 8:26 PM, "neelesh gadhia" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi Harish,
I downloaded the latest code from https://github.com/apache/hive/tree/ptf-windowing ( ptf-windowing branch).
I ran the same sql and get a different error now. Please advise.
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242020_0001, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242020_0001
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242020_0001
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-24 20:22:38,431 Stage-1 map = 0%, reduce = 0%
2013-02-24 20:22:40,451 Stage-1 map = 100%, reduce = 0%
2013-02-24 20:22:48,505 Stage-1 map = 100%, reduce = 33%
2013-02-24 20:22:52,530 Stage-1 map = 100%, reduce = 0%
2013-02-24
+
Butani, Harish 2013-02-25, 05:22
-
Re: [SQLWindowing] Windowing function output path syntax (#26)neelesh gadhia 2013-02-25, 06:01
treid 16 and 32 mb and get a different error.
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242150_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242150_0002
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242150_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-24 21:58:36,550 Stage-1 map = 0%, reduce = 0%
2013-02-24 21:58:38,560 Stage-1 map = 100%, reduce = 0%
2013-02-24 21:58:45,595 Stage-1 map = 100%, reduce = 33%
2013-02-24 21:58:48,616 Stage-1 map = 100%, reduce = 100%
2013-02-24 21:59:01,693 Stage-1 map = 100%, reduce = 0%
2013-02-24 21:59:08,723 Stage-1 map = 100%, reduce = 33%
2013-02-24 21:59:11,738 Stage-1 map = 100%, reduce = 100%
2013-02-24 21:59:25,805 Stage-1 map = 100%, reduce = 0%
2013-02-24 21:59:32,833 Stage-1 map = 100%, reduce = 33%
2013-02-24 21:59:35,852 Stage-1 map = 100%,
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242150_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242150_0002
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242150_0002
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2013-02-24 21:58:36,550 Stage-1 map = 0%, reduce = 0%
2013-02-24 21:58:38,560 Stage-1 map = 100%, reduce = 0%
2013-02-24 21:58:45,595 Stage-1 map = 100%, reduce = 33%
2013-02-24 21:58:48,616 Stage-1 map = 100%, reduce = 100%
2013-02-24 21:59:01,693 Stage-1 map = 100%, reduce = 0%
2013-02-24 21:59:08,723 Stage-1 map = 100%, reduce = 33%
2013-02-24 21:59:11,738 Stage-1 map = 100%, reduce = 100%
2013-02-24 21:59:25,805 Stage-1 map = 100%, reduce = 0%
2013-02-24 21:59:32,833 Stage-1 map = 100%, reduce = 33%
2013-02-24 21:59:35,852 Stage-1 map = 100%,
+
neelesh gadhia 2013-02-25, 06:01
-
Re: [SQLWindowing] Windowing function output path syntax (#26)Butani, Harish 2013-02-25, 17:53
How are you setting the value? It needs to be set in bytes.
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Sunday, February 24, 2013 10:01 PM
To: SAP SAP <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
treid 16 and 32 mb and get a different error.
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242150_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242150_0002
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242150_0002
Hadoop job information for Stage-1: number of map
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Sunday, February 24, 2013 10:01 PM
To: SAP SAP <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
treid 16 and 32 mb and get a different error.
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
set mapred.reduce.tasks=<number>
Starting Job = job_201302242150_0002, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201302242150_0002
Kill Command = /usr/local/Cellar/hadoop/1.1.1/libexec/bin/../bin/hadoop job -kill job_201302242150_0002
Hadoop job information for Stage-1: number of map
+
Butani, Harish 2013-02-25, 17:53
-
Re: [SQLWindowing] Windowing function output path syntax (#26)neelesh gadhia 2013-02-25, 18:55
my bad. After setting the value to 16MB in bytes (16777216) it worked fine.
Set hive.ptf.partition.persistence.memsize=16777216;
So does this mean default value of 64MB was causing issue? if yes, can you please elaborate a little how were you able to troubleshoot/debut the issue to this parameter setting?
Thanks,
Neelesh
________________________________
From: "Butani, Harish" <[EMAIL PROTECTED]>
To: neelesh gadhia <[EMAIL PROTECTED]>
Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Ashutosh Chauhan <[EMAIL PROTECTED]>
Sent: Monday, February 25, 2013 9:53 AM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
How are you setting the value? It needs to be set in bytes.
From: neelesh gadhia <[EMAIL PROTECTED]>
Reply-To: neelesh gadhia <[EMAIL PROTECTED]>
Date: Sunday, February 24, 2013 10:01 PM
To: SAP SAP <[EMAIL PROTECTED]>
Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
treid 16 and 32 mb and get a different error.
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per
Set hive.ptf.partition.persistence.memsize=16777216;
So does this mean default value of 64MB was causing issue? if yes, can you please elaborate a little how were you able to troubleshoot/debut the issue to this parameter setting?
Thanks,
Neelesh
________________________________
From: "Butani, Harish" <[EMAIL PROTECTED]>
To: neelesh gadhia <[EMAIL PROTECTED]>
Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; Ashutosh Chauhan <[EMAIL PROTECTED]>
Sent: Monday, February 25, 2013 9:53 AM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
How are you setting the value? It needs to be set in bytes.
From: neelesh gadhia <[EMAIL PROTECTED]>
Reply-To: neelesh gadhia <[EMAIL PROTECTED]>
Date: Sunday, February 24, 2013 10:01 PM
To: SAP SAP <[EMAIL PROTECTED]>
Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
treid 16 and 32 mb and get a different error.
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
6.SEL :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
7.FS :
RowResolver::
columns:[<null>._col0, <null>._col1, <null>._col2, <null>._col3]
Aliases:[
<null>:[mid -> _col0, tdate -> _col1, tamt -> _col2, com_sum -> _col3
]
columns mapped to expressions:[
]
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks not specified. Estimated from input data size: 1
In order to change the average load for a reducer (in bytes):
set hive.exec.reducers.bytes.per
+
neelesh gadhia 2013-02-25, 18:55
-
Re: [SQLWindowing] Windowing function output path syntax (#26)Butani, Harish 2013-02-26, 03:57
Seen this error before. It was pointed out by Ashutosh in Hive 896.
Haven't gotten around to changing the default…
To answer your question, the stack trace has:
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.hive.ql.exec.PTFPersistence$ByteBasedList.<init>(PTFPersistence.java:95)
...
Regards,
Harish
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Monday, February 25, 2013 10:55 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
my bad. After setting the value to 16MB in bytes (16777216) it worked fine.
Set hive.ptf.partition.persistence.memsize=16777216;
So does this mean default value of 64MB was causing issue? if yes, can you please elaborate a little how were you able to troubleshoot/debut the issue to this parameter setting?
Thanks,
Neelesh
________________________________
From: "Butani, Harish" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>; Ashutosh Chauhan <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Monday, February 25, 2013 9:53 AM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
How are you setting the value? It needs to be set in bytes.
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Sunday, February 24, 2013 10:01 PM
To: SAP SAP <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
treid 16 and 32 mb and get a different error.
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3,
Haven't gotten around to changing the default…
To answer your question, the stack trace has:
Caused by: java.lang.OutOfMemoryError: Java heap space
at org.apache.hadoop.hive.ql.exec.PTFPersistence$ByteBasedList.<init>(PTFPersistence.java:95)
...
Regards,
Harish
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Monday, February 25, 2013 10:55 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
my bad. After setting the value to 16MB in bytes (16777216) it worked fine.
Set hive.ptf.partition.persistence.memsize=16777216;
So does this mean default value of 64MB was causing issue? if yes, can you please elaborate a little how were you able to troubleshoot/debut the issue to this parameter setting?
Thanks,
Neelesh
________________________________
From: "Butani, Harish" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
To: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>; Ashutosh Chauhan <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Sent: Monday, February 25, 2013 9:53 AM
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
How are you setting the value? It needs to be set in bytes.
From: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: neelesh gadhia <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Sunday, February 24, 2013 10:01 PM
To: SAP SAP <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Cc: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>, "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: [SQLWindowing] Windowing function output path syntax (#26)
treid 16 and 32 mb and get a different error.
hive> Set hive.ptf.partition.persistence.memsize=16;
hive> select mid, tdate, tamt,sum(tamt) as com_sum over (rows between unbounded preceding and current row)
> from t_enc
> distribute by mid
> sort by mid, tdate;
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
:[(tok_function sum (tok_table_or_col tamt) (tok_windowspec (tok_windowrange (preceding unbounded) current))) -> com_sum
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
(TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current))) -> (TOK_FUNCTION sum (TOK_TABLE_OR_COL tamt) (TOK_WINDOWSPEC (TOK_WINDOWRANGE (preceding unbounded) current)))
]
1.TS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
2.SEL :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
3.RS :
RowResolver::
columns:[t_enc.mid, t_enc.tdate, t_enc.tamt, t_enc.BLOCK__OFFSET__INSIDE__FILE, t_enc.INPUT__FILE__NAME]
Aliases:[
t_enc:[mid -> mid, tdate -> tdate, tamt -> tamt, block__offset__inside__file -> BLOCK__OFFSET__INSIDE__FILE, input__file__name -> INPUT__FILE__NAME
]
columns mapped to expressions:[
]
4.EX :
RowResolver::
columns:[t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3, t_enc._col4]
Aliases:[
t_enc:[mid -> _col0, tdate -> _col1, tamt -> _col2, block__offset__inside__file -> _col3, input__file__name -> _col4
]
columns mapped to expressions:[
]
5.PTF :
RowResolver::
columns:[<null>.com_sum, t_enc._col0, t_enc._col1, t_enc._col2, t_enc._col3,
+
Butani, Harish 2013-02-26, 03:57
