Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> How to clone a logical plan ?


Copy link to this message
-
RE: How to clone a logical plan ?
If my memory serves me correctly, the logical plan cloning was
implemented (by me) for cloning inner plans for foreach. As such, the
top level plan cloning was never tested and some items are marked as
TODO (see visit methods for LOLoad, LOStore and LOStream).

If you want to use it as you mention in your test cases, then you need
to add code for cloning the LOLoad, LOStore, LOStream and LOJoin.

Santhosh
-----Original Message-----
From: Santhosh Srinivasan [mailto:[EMAIL PROTECTED]]
Sent: Thursday, November 05, 2009 4:04 PM
To: [EMAIL PROTECTED]
Subject: RE: How to clone a logical plan ?

You have hit a bug. I think LOJoin has to be added to
LogicalPlanCloneHelper.java. Can you file a jira?

Thanks,
Santhosh

-----Original Message-----
From: Ashutosh Chauhan [mailto:[EMAIL PROTECTED]]
Sent: Thursday, November 05, 2009 3:28 PM
To: [EMAIL PROTECTED]
Subject: How to clone a logical plan ?

Hi,

For our cost based optimizer for a given query plan we need to generate
alternative query plans and evaluate them based on their estimated cost.
As a result of that, I want to clone a logical plan. I thought
LogicalPlanCloner is meant for that, but it doesnt seem to work. I added
this simple test case in TestLogicalPlanBuilder.java

    public void testLogicalPlanCloneHelper() throws
CloneNotSupportedException{
        LogicalPlan lp = buildPlan("C = join ( load 'A') by $0, (load
'B') by $0;");
        LogicalPlanCloner cloner = new LogicalPlanCloner(lp);
        cloner.getClonedPlan();
    }

and this fails with the following stacktrace:

java.lang.NullPointerException
        at
org.apache.pig.impl.logicalLayer.LOVisitor.visit(LOVisitor.java:171)
        at
org.apache.pig.impl.logicalLayer.PlanSetter.visit(PlanSetter.java:63)
        at
org.apache.pig.impl.logicalLayer.LOJoin.visit(LOJoin.java:213)
        at org.apache.pig.impl.logicalLayer.LOJoin.visit(LOJoin.java:45)
        at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.ja
va:67)
        at
org.apache.pig.impl.plan.DepthFirstWalker.depthFirst(DepthFirstWalker.ja
va:69)
        at
org.apache.pig.impl.plan.DepthFirstWalker.walk(DepthFirstWalker.java:50)
        at
org.apache.pig.impl.plan.PlanVisitor.visit(PlanVisitor.java:51)
        at
org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.getClonedPlan(Lo
gicalPlanCloneHelper.java:73)
        at
org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(Logical
PlanCloner.java:46)
        at
org.apache.pig.test.TestLogicalPlanBuilder.testLogicalPlanCloneHelper(Te
stLogicalPlanBuilder.java:2110)

I am debugging this, but wanted to ask if I have hit a bug here or if I
am doing something wrong?

Thanks,
Ashutosh
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB