Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Problems using Pig with Oozie


Copy link to this message
-
Re: Problems using Pig with Oozie
+Oozie user list.

I don't see HbaseStorage related errors in the log. May be an Oozie
expert can point in the right direction.

Sent from my iPhone

On Feb 2, 2013, at 3:43 PM, Shawn Hermans <[EMAIL PROTECTED]> wrote:

> Thank you for your assistance.  I tried that and it did not work.  I looked
> at pig-0.10.0 and it looks like HBaseStorage should be included in the main
> project and is now longer in Piggybank.   Any other ideas? Is there an easy
> way I can see the Pig error messages? Looking at previous discussions, it
> looks like the only way to get to the original Pig error message is to
> write custom Java code to launch the Pig script.
>
>
> On Sat, Feb 2, 2013 at 2:50 PM, Jonas Hartwig <[EMAIL PROTECTED]>wrote:
>
>> You need to supply piggybank
>> <file>path/on/hdfs/piggybank.jar#piggybankjar</file>
>> And in the pig script
>> Register piggybankjar
>>
>> Jonas
>>
>> Shawn Hermans <[EMAIL PROTECTED]> schrieb:
>>
>>
>> All,
>> I have a Pig script that reads data from HBase using HBaseStorage, does
>> some manipulation with some Python UDFs and then writes it using
>> PigStorage.  It works fine when I run it as a standalone script, but will
>> not run in an Oozie workflow.  I can run normal Pig scripts using Oozie,
>> but run into problems when trying to run this script.  I believe I have
>> isolated the error to be with loading from HBaseStorage.  I stripped
>> everything out of my script except loading from HBaseStorage and outputting
>> to PigStorage.  The full script is below.
>>
>> profiles = LOAD 'hbase://profile' USING
>> org.apache.pig.backend.hadoop.hbase.HBaseStorage('e:*') as (columns:map[]);
>> limited = LIMIT profiles 200;
>> STORE limited into '/tmp/123456' using PigStorage();
>>
>> The log files are not very helpful.  It gives me an error Launcher ERROR,
>> reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2].
>> I also included the workflow and logfile below just in case.  Also, I am
>> running Cloudera 4.1.2.  I added all of the Oozie libraries to HDFS as
>> specified in the setup instructions.  I appreciate any help.
>>
>> Thanks,
>> Shawn
>>
>> <workflow-app xmlns="uri:oozie:workflow:0.3" name="simple-wf">
>>    <start to="pig-node"/>
>>    <action name="pig-node">
>>        <pig>
>>            <job-tracker>${jobTracker}</job-tracker>
>>            <name-node>${nameNode}</name-node>
>>            <prepare>
>>                <delete path="${nameNode}/user/${wf:user()}/tmp/65321"/>
>>            </prepare>
>>            <configuration>
>>                <property>
>>                    <name>mapred.job.queue.name</name>
>>                    <value>${queueName}</value>
>>                </property>
>>                <property>
>>                    <name>mapred.compress.map.output</name>
>>                    <value>true</value>
>>                </property>
>>            </configuration>
>>            <script>simple.pig</script>
>>        </pig>
>>        <ok to="end"/>
>>        <error to="fail"/>
>>    </action>
>>    <kill name="fail">
>>        <message>Job failed,error
>> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>>    </kill>
>>    <end name="end"/>
>> </workflow-app>
>>
>>
>>
>> 2013-02-02 14:42:50,123 INFO
>> org.apache.oozie.command.wf.ActionStartXCommand: USER[dev] GROUP[-] TOKEN[]
>> APP[simple-wf] JOB[0000035-130130192939212-oozie-oozi-W]
>> ACTION[0000035-130130192939212-oozie-oozi-W@:start:] Start action
>> [0000035-130130192939212-oozie-oozi-W@:start:] with user-retry state :
>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> 2013-02-02 14:42:50,123 WARN
>> org.apache.oozie.command.wf.ActionStartXCommand: USER[dev] GROUP[-] TOKEN[]
>> APP[simple-wf] JOB[0000035-130130192939212-oozie-oozi-W]
>> ACTION[0000035-130130192939212-oozie-oozi-W@:start:]
>> [***0000035-130130192939212-oozie-oozi-W@:start:***]Action status=DONE
>> 2013-02-02 14:42:50,123 WARN
>> org.apache.oozie.command.wf.ActionStartXCommand: USER[dev] GROUP[-] TOKEN[]
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB