Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> illustrate


Copy link to this message
-
Re: illustrate
Hi to everybody!

Thanks Thejas for your reply.

Current situation is the following:

Rank command is composed by two physical operators connected: POCounter and
PORank.

This operator has two implementations:
* The first one is composed by two map-only jobs (one per each physical
operator of rank command). It was quite easy to incorporate the illustrator
command on it.

* The second implementation is a chain of physical operators, and at the
end are the POCounter, PORank and a POForEach. And I have some problems on
illustrate it.
This is what I got, while trying to illustrate here:

java.lang.NullPointerException

at org.apache.pig.pen.util.LineageTracer.link(LineageTracer.java:70)

at org.apache.pig.pen.util.LineageTracer.union(LineageTracer.java:56)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.illustratorMarkup(
POForEach.java:743)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.createTuple(
POForEach.java:488)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(
POForEach.java:436)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(
POForEach.java:294)

at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(
PigGenericMapBase.java:275)

at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(
PigGenericMapBase.java:270)

at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(
PigGenericMapBase.java:1)
While I was looking for a solution, I think one possible reason is the
LineageTracer used at POForEach, which should be implemented at PORank too.
(But I don't know how to used it)

Thanks in advance for your reply,

--

Allan AvendaƱo S.
Computer Engineer
Ex-SWY22 Participant
Rome - Italy
Gmail: [EMAIL PROTECTED]
--
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB