Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Problems using Pig with Oozie


Copy link to this message
-
Re: Problems using Pig with Oozie
Prashant Kommireddi 2013-02-02, 22:21
+Oozie user list.

I don't see HbaseStorage related errors in the log. May be an Oozie
expert can point in the right direction.

Sent from my iPhone

On Feb 2, 2013, at 3:43 PM, Shawn Hermans <[EMAIL PROTECTED]> wrote:

> Thank you for your assistance.  I tried that and it did not work.  I looked
> at pig-0.10.0 and it looks like HBaseStorage should be included in the main
> project and is now longer in Piggybank.   Any other ideas? Is there an easy
> way I can see the Pig error messages? Looking at previous discussions, it
> looks like the only way to get to the original Pig error message is to
> write custom Java code to launch the Pig script.
>
>
> On Sat, Feb 2, 2013 at 2:50 PM, Jonas Hartwig <[EMAIL PROTECTED]>wrote:
>
>> You need to supply piggybank
>> <file>path/on/hdfs/piggybank.jar#piggybankjar</file>
>> And in the pig script
>> Register piggybankjar
>>
>> Jonas
>>
>> Shawn Hermans <[EMAIL PROTECTED]> schrieb:
>>
>>
>> All,
>> I have a Pig script that reads data from HBase using HBaseStorage, does
>> some manipulation with some Python UDFs and then writes it using
>> PigStorage.  It works fine when I run it as a standalone script, but will
>> not run in an Oozie workflow.  I can run normal Pig scripts using Oozie,
>> but run into problems when trying to run this script.  I believe I have
>> isolated the error to be with loading from HBaseStorage.  I stripped
>> everything out of my script except loading from HBaseStorage and outputting
>> to PigStorage.  The full script is below.
>>
>> profiles = LOAD 'hbase://profile' USING
>> org.apache.pig.backend.hadoop.hbase.HBaseStorage('e:*') as (columns:map[]);
>> limited = LIMIT profiles 200;
>> STORE limited into '/tmp/123456' using PigStorage();
>>
>> The log files are not very helpful.  It gives me an error Launcher ERROR,
>> reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2].
>> I also included the workflow and logfile below just in case.  Also, I am
>> running Cloudera 4.1.2.  I added all of the Oozie libraries to HDFS as
>> specified in the setup instructions.  I appreciate any help.
>>
>> Thanks,
>> Shawn
>>
>> <workflow-app xmlns="uri:oozie:workflow:0.3" name="simple-wf">
>>    <start to="pig-node"/>
>>    <action name="pig-node">
>>        <pig>
>>            <job-tracker>${jobTracker}</job-tracker>
>>            <name-node>${nameNode}</name-node>
>>            <prepare>
>>                <delete path="${nameNode}/user/${wf:user()}/tmp/65321"/>
>>            </prepare>
>>            <configuration>
>>                <property>
>>                    <name>mapred.job.queue.name</name>
>>                    <value>${queueName}</value>
>>                </property>
>>                <property>
>>                    <name>mapred.compress.map.output</name>
>>                    <value>true</value>
>>                </property>
>>            </configuration>
>>            <script>simple.pig</script>
>>        </pig>
>>        <ok to="end"/>
>>        <error to="fail"/>
>>    </action>
>>    <kill name="fail">
>>        <message>Job failed,error
>> message[${wf:errorMessage(wf:lastErrorNode())}]</message>
>>    </kill>
>>    <end name="end"/>
>> </workflow-app>
>>
>>
>>
>> 2013-02-02 14:42:50,123 INFO
>> org.apache.oozie.command.wf.ActionStartXCommand: USER[dev] GROUP[-] TOKEN[]
>> APP[simple-wf] JOB[0000035-130130192939212-oozie-oozi-W]
>> ACTION[0000035-130130192939212-oozie-oozi-W@:start:] Start action
>> [0000035-130130192939212-oozie-oozi-W@:start:] with user-retry state :
>> userRetryCount [0], userRetryMax [0], userRetryInterval [10]
>> 2013-02-02 14:42:50,123 WARN
>> org.apache.oozie.command.wf.ActionStartXCommand: USER[dev] GROUP[-] TOKEN[]
>> APP[simple-wf] JOB[0000035-130130192939212-oozie-oozi-W]
>> ACTION[0000035-130130192939212-oozie-oozi-W@:start:]
>> [***0000035-130130192939212-oozie-oozi-W@:start:***]Action status=DONE
>> 2013-02-02 14:42:50,123 WARN
>> org.apache.oozie.command.wf.ActionStartXCommand: USER[dev] GROUP[-] TOKEN[]