Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Exception: Unable to find clone for op Project 4-16 Projections


Copy link to this message
-
Re: Exception: Unable to find clone for op Project 4-16 Projections
Ashutosh Chauhan 2010-05-18, 19:34
> we do have multiple threads,
This is what I suspected from beginning :)

PigServer is not thread-safe, yet.

Ashutosh

On Tue, May 18, 2010 at 12:04, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
> seems it is a known issue. we do have multiple threads, each thread
> will take one script and new a PigServer to run it...
>
> Thanks,
> Yonggang
>
> On Mon, May 17, 2010 at 6:06 PM, Ashutosh Chauhan
> <[EMAIL PROTECTED]> wrote:
>> From Yonggang description and code snippet .. it seems to me he is not
>> having multithreaded environment.. there is only one thread and he is
>> creating new PigServer instance in it repeatedly for each query... and
>> since static variables are reset everytime.. this should work...
>> PigServer still doesnt work in multi-threaded environment...
>>
>> Ashutosh
>>
>> On Mon, May 17, 2010 at 18:00, Jeff Zhang <[EMAIL PROTECTED]> wrote:
>>> Ashutosh,
>>>
>>> Are you sure pig now can work on multi-thread environment ? As I know
>>> pig 0.5 can not work on multi-thread environment even you are create
>>> new PigServer for each pig script.
>>>
>>>
>>> On Tue, May 18, 2010 at 5:43 AM, Ashutosh Chauhan
>>> <[EMAIL PROTECTED]> wrote:
>>>> if you are creating new instance for each query.. you should be fine..
>>>> which Pig version are you using ? can you paste the snippet of java
>>>> code where you are creating new pig server instance and then using it
>>>> for a new query ?
>>>>
>>>> Ashutosh
>>>> On Mon, May 17, 2010 at 14:17, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>> sorry, actually yes. no, we use new instance for each script.
>>>>>
>>>>> On Mon, May 17, 2010 at 2:13 PM, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>> yes. yes.
>>>>>>
>>>>>> On Mon, May 17, 2010 at 2:03 PM, Ashutosh Chauhan
>>>>>> <[EMAIL PROTECTED]> wrote:
>>>>>>> Are you using PigServer java api to run your Pig queries ? If so, are
>>>>>>> you trying to run multiple queries in different threads against same
>>>>>>> Pig server  instance?
>>>>>>>
>>>>>>> Ashutosh
>>>>>>>
>>>>>>> On Mon, May 17, 2010 at 13:57, Yonggang Qiao <[EMAIL PROTECTED]> wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> anyone has seen this error before? normally our script runs fine, but
>>>>>>>> sometime recently it began to throw this exception. also usually it
>>>>>>>> will go away if I rerun it.
>>>>>>>>
>>>>>>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
>>>>>>>> Project 4-16 Projections: [9] Overloaded: false
>>>>>>>>        at org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:132)
>>>>>>>>        at org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:66)
>>>>>>>>        at org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>>>>>>        at org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3009)
>>>>>>>>        at org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1314)
>>>>>>>>        at org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:893)
>>>>>>>>        at org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:682)
>>>>>>>>        at org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>>>>>>        at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1017)
>>>>>>>>        at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:967)
>>>>>>>>        at org.apache.pig.PigServer.registerQuery(PigServer.java:383)
>>>>>>>>        at org.apache.pig.PigServer.registerQuery(PigServer.java:399)
>>>>>>>>        at ...
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Yonggang
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards
>>>
>>> Jeff Zhang
>>>
>>
>