Pig, mail # user - Re: Exception: Unable to find clone for op Project 4-16 Projections

Exception: Unable to find clone for op Project 4-16 Projections

Yonggang Qiao 2010-05-17, 20:57

Hi,

anyone has seen this error before? normally our script runs fine, but
sometime recently it began to throw this exception. also usually it
will go away if I rerun it.

Caused by: java.lang.RuntimeException: Unable to find clone for op
Project 4-16 Projections: [9] Overloaded: false
at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
at ...

Thanks,
Yonggang

Re: Exception: Unable to find clone for op Project 4-16 Projections

Ashutosh Chauhan 2010-05-17, 21:03

Are you using PigServer java api to run your Pig queries ? If so, are
you trying to run multiple queries in different threads against same
Pig server instance?

Ashutosh

On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
> Hi,
>
> anyone has seen this error before? normally our script runs fine, but
> sometime recently it began to throw this exception. also usually it
> will go away if I rerun it.
>
> Caused by: java.lang.RuntimeException: Unable to find clone for op
> Project 4-16 Projections: [9] Overloaded: false
> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
> at ...
>
> Thanks,
> Yonggang
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Yonggang Qiao 2010-05-17, 21:13

yes. yes.

On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
<[EMAIL PROTECTED]> wrote:
> Are you using PigServer java api to run your Pig queries ? If so, are
> you trying to run multiple queries in different threads against same
> Pig server instance?
>
> Ashutosh
>
> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> anyone has seen this error before? normally our script runs fine, but
>> sometime recently it began to throw this exception. also usually it
>> will go away if I rerun it.
>>
>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>> Project 4-16 Projections: [9] Overloaded: false
>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>> at ...
>>
>> Thanks,
>> Yonggang
>>
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Yonggang Qiao 2010-05-17, 21:17

sorry, actually yes. no, we use new instance for each script.

On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
> yes. yes.
>
> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
> <[EMAIL PROTECTED]> wrote:
>> Are you using PigServer java api to run your Pig queries ? If so, are
>> you trying to run multiple queries in different threads against same
>> Pig server instance?
>>
>> Ashutosh
>>
>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>> Hi,
>>>
>>> anyone has seen this error before? normally our script runs fine, but
>>> sometime recently it began to throw this exception. also usually it
>>> will go away if I rerun it.
>>>
>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>> Project 4-16 Projections: [9] Overloaded: false
>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>> at ...
>>>
>>> Thanks,
>>> Yonggang
>>>
>>
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Ashutosh Chauhan 2010-05-17, 21:43

if you are creating new instance for each query.. you should be fine..
which Pig version are you using ? can you paste the snippet of java
code where you are creating new pig server instance and then using it
for a new query ?

Ashutosh
On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
> sorry, actually yes. no, we use new instance for each script.
>
> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>> yes. yes.
>>
>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>> <[EMAIL PROTECTED]> wrote:
>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>> you trying to run multiple queries in different threads against same
>>> Pig server instance?
>>>
>>> Ashutosh
>>>
>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>> Hi,
>>>>
>>>> anyone has seen this error before? normally our script runs fine, but
>>>> sometime recently it began to throw this exception. also usually it
>>>> will go away if I rerun it.
>>>>
>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>> Project 4-16 Projections: [9] Overloaded: false
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>> at ...
>>>>
>>>> Thanks,
>>>> Yonggang
>>>>
>>>
>>
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Yonggang Qiao 2010-05-17, 21:58

sure. removed the irrelevant stuff.

PigServer pigServer = new PigServer("mapreduce");

pigServer.registerQuery(...);
...

ExecJob job = pigServer.store("RET", outputPath);

then periodically check the job.getStatus().

On Mon, May 17, 2010 at 2:43 PM, Ashutosh Chauhan
<[EMAIL PROTECTED]> wrote:
> if you are creating new instance for each query.. you should be fine..
> which Pig version are you using ? can you paste the snippet of java
> code where you are creating new pig server instance and then using it
> for a new query ?
>
> Ashutosh
> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>> sorry, actually yes. no, we use new instance for each script.
>>
>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>> yes. yes.
>>>
>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>> <[EMAIL PROTECTED]> wrote:
>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>> you trying to run multiple queries in different threads against same
>>>> Pig server instance?
>>>>
>>>> Ashutosh
>>>>
>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>> Hi,
>>>>>
>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>> sometime recently it began to throw this exception. also usually it
>>>>> will go away if I rerun it.
>>>>>
>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>> at ...
>>>>>
>>>>> Thanks,
>>>>> Yonggang
>>>>>
>>>>
>>>
>>
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Jeff Zhang 2010-05-18, 01:00

Ashutosh,

Are you sure pig now can work on multi-thread environment ? As I know
pig 0.5 can not work on multi-thread environment even you are create
new PigServer for each pig script.
On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
<[EMAIL PROTECTED]> wrote:
> if you are creating new instance for each query.. you should be fine..
> which Pig version are you using ? can you paste the snippet of java
> code where you are creating new pig server instance and then using it
> for a new query ?
>
> Ashutosh
> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>> sorry, actually yes. no, we use new instance for each script.
>>
>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>> yes. yes.
>>>
>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>> <[EMAIL PROTECTED]> wrote:
>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>> you trying to run multiple queries in different threads against same
>>>> Pig server instance?
>>>>
>>>> Ashutosh
>>>>
>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>> Hi,
>>>>>
>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>> sometime recently it began to throw this exception. also usually it
>>>>> will go away if I rerun it.
>>>>>
>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>> at ...
>>>>>
>>>>> Thanks,
>>>>> Yonggang
>>>>>
>>>>
>>>
>>
>

--
Best Regards

Jeff Zhang

Re: Exception: Unable to find clone for op Project 4-16 Projections

Ashutosh Chauhan 2010-05-18, 01:06

>From Yonggang description and code snippet .. it seems to me he is not
having multithreaded environment.. there is only one thread and he is
creating new PigServer instance in it repeatedly for each query... and
since static variables are reset everytime.. this should work...
PigServer still doesnt work in multi-threaded environment...

Ashutosh

On Mon, May 17, 2010 at 18:00, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> Ashutosh,
>
> Are you sure pig now can work on multi-thread environment ? As I know
> pig 0.5 can not work on multi-thread environment even you are create
> new PigServer for each pig script.
>
>
> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
> <[EMAIL PROTECTED]> wrote:
>> if you are creating new instance for each query.. you should be fine..
>> which Pig version are you using ? can you paste the snippet of java
>> code where you are creating new pig server instance and then using it
>> for a new query ?
>>
>> Ashutosh
>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>> sorry, actually yes. no, we use new instance for each script.
>>>
>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>> yes. yes.
>>>>
>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>> you trying to run multiple queries in different threads against same
>>>>> Pig server instance?
>>>>>
>>>>> Ashutosh
>>>>>
>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>> will go away if I rerun it.
>>>>>>
>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>> at ...
>>>>>>
>>>>>> Thanks,
>>>>>> Yonggang
>>>>>>
>>>>>
>>>>
>>>
>>
>
>
>
> --
> Best Regards
>
> Jeff Zhang
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Yonggang Qiao 2010-05-18, 19:04

seems it is a known issue. we do have multiple threads, each thread
will take one script and new a PigServer to run it...

Thanks,
Yonggang

On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
<[EMAIL PROTECTED]> wrote:
> From Yonggang description and code snippet .. it seems to me he is not
> having multithreaded environment.. there is only one thread and he is
> creating new PigServer instance in it repeatedly for each query... and
> since static variables are reset everytime.. this should work...
> PigServer still doesnt work in multi-threaded environment...
>
> Ashutosh
>
> On Mon, May 17, 2010 at 18:00, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>> Ashutosh,
>>
>> Are you sure pig now can work on multi-thread environment ? As I know
>> pig 0.5 can not work on multi-thread environment even you are create
>> new PigServer for each pig script.
>>
>>
>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>> <[EMAIL PROTECTED]> wrote:
>>> if you are creating new instance for each query.. you should be fine..
>>> which Pig version are you using ? can you paste the snippet of java
>>> code where you are creating new pig server instance and then using it
>>> for a new query ?
>>>
>>> Ashutosh
>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>> sorry, actually yes. no, we use new instance for each script.
>>>>
>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>> yes. yes.
>>>>>
>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>> you trying to run multiple queries in different threads against same
>>>>>> Pig server instance?
>>>>>>
>>>>>> Ashutosh
>>>>>>
>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>> will go away if I rerun it.
>>>>>>>
>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>> at ...
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Yonggang
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Best Regards
>>
>> Jeff Zhang
>>
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Ashutosh Chauhan 2010-05-18, 19:34

> we do have multiple threads,
This is what I suspected from beginning :)

PigServer is not thread-safe, yet.

Ashutosh

On Tue, May 18, 2010 at 12:04, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
> seems it is a known issue. we do have multiple threads, each thread
> will take one script and new a PigServer to run it...
>
> Thanks,
> Yonggang
>
> On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
> <[EMAIL PROTECTED]> wrote:
>> From Yonggang description and code snippet .. it seems to me he is not
>> having multithreaded environment.. there is only one thread and he is
>> creating new PigServer instance in it repeatedly for each query... and
>> since static variables are reset everytime.. this should work...
>> PigServer still doesnt work in multi-threaded environment...
>>
>> Ashutosh
>>
>> On Mon, May 17, 2010 at 18:00, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>> Ashutosh,
>>>
>>> Are you sure pig now can work on multi-thread environment ? As I know
>>> pig 0.5 can not work on multi-thread environment even you are create
>>> new PigServer for each pig script.
>>>
>>>
>>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>>> <[EMAIL PROTECTED]> wrote:
>>>> if you are creating new instance for each query.. you should be fine..
>>>> which Pig version are you using ? can you paste the snippet of java
>>>> code where you are creating new pig server instance and then using it
>>>> for a new query ?
>>>>
>>>> Ashutosh
>>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>> sorry, actually yes. no, we use new instance for each script.
>>>>>
>>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>> yes. yes.
>>>>>>
>>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>>> you trying to run multiple queries in different threads against same
>>>>>>> Pig server instance?
>>>>>>>
>>>>>>> Ashutosh
>>>>>>>
>>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>>> will go away if I rerun it.
>>>>>>>>
>>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>>> at ...
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Yonggang
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>

Re: Exception: Unable to find clone for op Project 4-16 Projections

Jeff Zhang 2010-05-19, 00:04

And you can refer here http://issues.apache.org/jira/browse/PIG-240
People has done some work for this issue although it is still not
resolved completely.

On Tue, May 18, 2010 at 12:34 PM, Ashutosh Chauhan
<[EMAIL PROTECTED]> wrote:
>> we do have multiple threads,
> This is what I suspected from beginning :)
>
> PigServer is not thread-safe, yet.
>
> Ashutosh
>
> On Tue, May 18, 2010 at 12:04, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>> seems it is a known issue. we do have multiple threads, each thread
>> will take one script and new a PigServer to run it...
>>
>> Thanks,
>> Yonggang
>>
>> On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
>> <[EMAIL PROTECTED]> wrote:
>>> From Yonggang description and code snippet .. it seems to me he is not
>>> having multithreaded environment.. there is only one thread and he is
>>> creating new PigServer instance in it repeatedly for each query... and
>>> since static variables are reset everytime.. this should work...
>>> PigServer still doesnt work in multi-threaded environment...
>>>
>>> Ashutosh
>>>
>>> On Mon, May 17, 2010 at 18:00, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>>> Ashutosh,
>>>>
>>>> Are you sure pig now can work on multi-thread environment ? As I know
>>>> pig 0.5 can not work on multi-thread environment even you are create
>>>> new PigServer for each pig script.
>>>>
>>>>
>>>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>>>> <[EMAIL PROTECTED]> wrote:
>>>>> if you are creating new instance for each query.. you should be fine..
>>>>> which Pig version are you using ? can you paste the snippet of java
>>>>> code where you are creating new pig server instance and then using it
>>>>> for a new query ?
>>>>>
>>>>> Ashutosh
>>>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>> sorry, actually yes. no, we use new instance for each script.
>>>>>>
>>>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>> yes. yes.
>>>>>>>
>>>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>>>> you trying to run multiple queries in different threads against same
>>>>>>>> Pig server instance?
>>>>>>>>
>>>>>>>> Ashutosh
>>>>>>>>
>>>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>>>> Hi,
>>>>>>>>>
>>>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>>>> will go away if I rerun it.
>>>>>>>>>
>>>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>>>> at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>>>> at ...
>>>>>>>>>
>>>>>>>>
Best Regards

Jeff Zhang

Re: Exception: Unable to find clone for op Project 4-16 Projections

Yonggang Qiao 2010-05-19, 20:04

will definitely give it a try. Thanks!

On Tue, May 18, 2010 at 5:04 PM, Jeff Zhang <[EMAIL PROTECTED]> wrote:
> And you can refer here http://issues.apache.org/jira/browse/PIG-240
> People has done some work for this issue although it is still not
> resolved completely.
>
>
>
> On Tue, May 18, 2010 at 12:34 PM, Ashutosh Chauhan
> <[EMAIL PROTECTED]> wrote:
>>> we do have multiple threads,
>> This is what I suspected from beginning :)
>>
>> PigServer is not thread-safe, yet.
>>
>> Ashutosh
>>
>> On Tue, May 18, 2010 at 12:04, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>> seems it is a known issue. we do have multiple threads, each thread
>>> will take one script and new a PigServer to run it...
>>>
>>> Thanks,
>>> Yonggang
>>>
>>> On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
>>> <[EMAIL PROTECTED]> wrote:
>>>> From Yonggang description and code snippet .. it seems to me he is not
>>>> having multithreaded environment.. there is only one thread and he is
>>>> creating new PigServer instance in it repeatedly for each query... and
>>>> since static variables are reset everytime.. this should work...
>>>> PigServer still doesnt work in multi-threaded environment...
>>>>
>>>> Ashutosh
>>>>
>>>> On Mon, May 17, 2010 at 18:00, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>>>> Ashutosh,
>>>>>
>>>>> Are you sure pig now can work on multi-thread environment ? As I know
>>>>> pig 0.5 can not work on multi-thread environment even you are create
>>>>> new PigServer for each pig script.
>>>>>
>>>>>
>>>>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>> if you are creating new instance for each query.. you should be fine..
>>>>>> which Pig version are you using ? can you paste the snippet of java
>>>>>> code where you are creating new pig server instance and then using it
>>>>>> for a new query ?
>>>>>>
>>>>>> Ashutosh
>>>>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>> sorry, actually yes. no, we use new instance for each script.
>>>>>>>
>>>>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>>> yes. yes.
>>>>>>>>
>>>>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>>>>> you trying to run multiple queries in different threads against same
>>>>>>>>> Pig server instance?
>>>>>>>>>
>>>>>>>>> Ashutosh
>>>>>>>>>
>>>>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>>>>> will go away if I rerun it.
>>>>>>>>>>
>>>>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>>>>> at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>>>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>>>>> at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>