Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> illustrate


+
Gianmarco De Francisci Mo... 2012-08-02, 13:52
+
Thejas Nair 2012-08-03, 21:30
+
Gianmarco De Francisci Mo... 2012-08-04, 07:22
Copy link to this message
-
Re: illustrate
Hi to everybody!

Thanks Thejas for your reply.

Current situation is the following:

Rank command is composed by two physical operators connected: POCounter and
PORank.

This operator has two implementations:
* The first one is composed by two map-only jobs (one per each physical
operator of rank command). It was quite easy to incorporate the illustrator
command on it.

* The second implementation is a chain of physical operators, and at the
end are the POCounter, PORank and a POForEach. And I have some problems on
illustrate it.
This is what I got, while trying to illustrate here:

java.lang.NullPointerException

at org.apache.pig.pen.util.LineageTracer.link(LineageTracer.java:70)

at org.apache.pig.pen.util.LineageTracer.union(LineageTracer.java:56)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.illustratorMarkup(
POForEach.java:743)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.createTuple(
POForEach.java:488)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.processPlan(
POForEach.java:436)

at
org.apache.pig.backend.hadoop.executionengine.physicalLayer.relationalOperators.POForEach.getNext(
POForEach.java:294)

at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(
PigGenericMapBase.java:275)

at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(
PigGenericMapBase.java:270)

at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(
PigGenericMapBase.java:1)
While I was looking for a solution, I think one possible reason is the
LineageTracer used at POForEach, which should be implemented at PORank too.
(But I don't know how to used it)

Thanks in advance for your reply,

--

Allan AvendaƱo S.
Computer Engineer
Ex-SWY22 Participant
Rome - Italy
Gmail: [EMAIL PROTECTED]
--
+
Allan 2012-08-07, 14:34
+
Gianmarco De Francisci Mo... 2012-08-09, 16:01
+
Dmitriy Ryaboy 2012-08-10, 07:13
+
Gianmarco De Francisci Mo... 2012-08-12, 09:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB