Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Failure to run pig jobs using HbaseStorage in Oozie


Copy link to this message
-
Re: Failure to run pig jobs using HbaseStorage in Oozie
Rohini Palaniswamy 2013-03-15, 15:08
Pig 3206 fixed an issue when the hbase cluster was secure. You must be
facing a different issue. Need to see the stack trace in the hadoop job log
for the actual error. Click on your workflow in oozie UI and it will show
the tracker url to the actual hadoop job. Click on it and look at the
syslog of the map task.
On Mar 14, 2013 9:20 PM, "Praveen Bysani" <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Following is the configuration from my core-site and hbase-site.xml,
>
>   <property>
>      <name>hadoop.security.
> authentication</name>
>     <value>simple</value>
>   </property>
>   <property>
>      <name>hadoop.rpc.protection</name>
>      <value>authentication</value>
>    </property>
>    <property>
>      <name>hadoop.security.auth_to_local</name>
>      <value>DEFAULT</value>
>    </property>
>
> So i guess i may not be using a secure hadoop/hbase. I am not sure what you
> meant by the log of pig launcher job of hadoop oozie. Do you mean the log
> in Job Tracker for this job id ?
>
>
> On 15 March 2013 04:43, Rohini Palaniswamy <[EMAIL PROTECTED]>
> wrote:
>
> > Hi Praveen,
> >    Are you running a secure cluster - secure hadoop and hbase? Can you
> > check what is the stacktrace on the pig launcher job log of Hadoop Oozie?
> >
> > Regards,
> > Rohini
> >
> >
> > On Thu, Mar 14, 2013 at 2:28 AM, Praveen Bysani <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi,
> > >
> > > I am trying to run a simple pig script that uses HbaseStorage class to
> > load
> > > data from a hbase table. The pig script runs perfectly fine when run
> > > standalone in mapreduce mode. But when i submit it as a action in oozie
> > > workflow, the job always fails. The oozie job log for that workflow
> gives
> > > following errrors which is not very useful,
> > >
> > > JOB[0000004-130312101540251-oozie-oozi-W]
> > > ACTION[0000004-130312101540251-oozie-oozi-W@pig-node] action
> completed,
> > > external ID [job_201303111007_0043]
> > > 2013-03-14 08:41:22,658 WARN
> > > org.apache.oozie.action.hadoop.PigActionExecutor: USER[hadoopuser]
> > GROUP[-]
> > > TOKEN[] APP[pig-wf] JOB[0000004-130312101540251-oozie-oozi-W]
> > > ACTION[0000004-130312101540251-oozie-oozi-W@pig-node] *Launcher
> > > ERROR,*reason: Main class
> > > *[org.apache.oozie.action.hadoop.PigMain], exit code [2]*
> > > 2013-03-14 08:41:22,833 INFO
> > org.apache.oozie.command.wf.ActionEndXCommand:
> > > USER[hadoopuser] GROUP[-] TOKEN[] APP[pig-wf]
> > > JOB[0000004-130312101540251-oozie-oozi-W]
> > > ACTION[0000004-130312101540251-oozie-oozi-W@pig-node] ERROR is
> > considered
> > > as FAILED for SLA
> > > 2013-03-14 08:41:22,892 INFO
> > > org.apache.oozie.command.wf.ActionStartXCommand: USER[hadoopuser]
> > GROUP[-]
> > > TOKEN[] APP[pig-wf] JOB[0000004-130312101540251-oozie-oozi-W]
> > > ACTION[0000004-130312101540251-oozie-oozi-W@fail] Start action
> > > [0000004-130312101540251-oozie-oozi-W@fail] with user-retry state :
> > > userRetryCount [0], userRetryMax [0], userRetryInterval [10]
> > > 2013-03-14 08:41:22,893 WARN
> > > org.apache.oozie.command.wf.ActionStartXCommand: USER[hadoopuser]
> > GROUP[-]
> > > TOKEN[] APP[pig-wf] JOB[0000004-130312101540251-oozie-oozi-W]
> > > ACTION[0000004-130312101540251-oozie-oozi-W@fail]
> > > [***0000004-130312101540251-oozie-oozi-W@fail***]Action status=DONE
> > > 2013-03-14 08:41:22,893 WARN
> > > org.apache.oozie.command.wf.ActionStartXCommand: USER[hadoopuser]
> > GROUP[-]
> > > TOKEN[] APP[pig-wf] JOB[0000004-130312101540251-oozie-oozi-W]
> > > ACTION[0000004-130312101540251-oozie-oozi-W@fail]
> > > [***0000004-130312101540251-oozie-oozi-W@fail***]Action updated in DB!
> > > 2013-03-14 08:41:22,997 WARN
> > > org.apache.oozie.command.coord.CoordActionUpdateXCommand:
> > USER[hadoopuser]
> > > GROUP[-] TOKEN[] APP[pig-wf] JOB[0000004-130312101540251-oozie-oozi-W]
> > > ACTION[-] E1100: Command precondition does not hold before execution,
> [,
> > > coord action is null], Error Code: E1100
> > >
> > >
> > > I did some searching and found that a patch has been deployed for this