Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Pig on standalone hadoop 19.1


Copy link to this message
-
RE: Pig on standalone hadoop 19.1
Hi Lance,

Sorry for this long exchange. Let me make sure I understand.

You have a pig.jar that works fine on Hadoop 18. When you apply patch
for hadoop19 to the same code and run identical job (with identical
command line) against hadoop 19 cluster. You see the problem with
connecting to HOD. Is this correct?

Did you try, as Alan suggested, adding the property on the command line
just to see if it solves the issue?

The properties file that is listed below is the one you included in
pig.jar, right?

I know that Pradeep who created the patch ran it against 1-node Hadoop
19 cluster and did not see this issue so I suspect it is somehow related
to your setup.

If we can't figure out this over email, I will setup Hadoop 19 cluster
over the weekend and try it out.

Thanks,

Olga

> -----Original Message-----
> From: Lance Riedel [mailto:[EMAIL PROTECTED]]
> Sent: Monday, May 04, 2009 10:10 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Pig on standalone hadoop 19.1
>
> Hi Olga,
> I haven't had any problems on 18. Unfortunately our cluster
> is on 19.1 for many other reasons (I don't have them all),
> because Pig is not our primary requirement.
>
> Pig is something we are wanting to do to do some more
> analytics/ reporting, but it is after the fact of all of our
> indexing, data processing needs already in place on 19.1.
>
> Thanks,
> Lance
> On May 2, 2009, at 4:21 PM, Olga Natkovich wrote:
>
> > Hi Lance,
> >
> > Is there a way you can use Pig with Hadoop 18 cluster? Most of our
> > testing at this point is done on 18 so we have much more experience
> > with that.
> >
> > Olga
> >
> >> -----Original Message-----
> >> From: Lance Riedel [mailto:[EMAIL PROTECTED]]
> >> Sent: Saturday, May 02, 2009 9:09 AM
> >> To: [EMAIL PROTECTED]
> >> Subject: Re: Pig on standalone hadoop 19.1
> >>
> >> Hi Alan,
> >> Thanks - I had put that value in my pig.properties (see the very
> >> bottom - below). But it didn't seem to help there- maybe I'm not
> >> configuring the use of the properties file correctly??
> >>
> >>
> >> The command line option DOES work however, so I'm going to
> it to my
> >> scripts as a work around for now.
> >>
> >> Thanks,
> >> Lance
> >>
> >> On May 1, 2009, at 10:40 AM, Alan Gates wrote:
> >>
> >>> Can you try invoking pig with -Dhod.server="".  If that
> works, you
> >>> should be able to set that value in your pig.properties file.
> >>>
> >>> Alan.
> >>>
> >>> On Apr 29, 2009, at 2:02 PM, Lance Riedel wrote:
> >>>
> >>>> Hi all,
> >>>> I'm having issues trying to run pig on a stand alone
> >> hadoop cluster.
> >>>> The cluster is running 19.1, but I have applied the
> >> following patch:
> >>>> http://issues.apache.org/jira/browse/PIG-573
> >>>>
> >>>> When I start pig, I get the following  (infinite loop):
> >>>>
> >>>> 2009-04-29 16:55:27,220 [main] WARN
> >>>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> >>>> Failed to create HOD configuration directory - /tmp/
> >>>>
> PigHod.domU-12-31-38-00-C4-31.dotspots.3204342197969613Retrying...
> >>>> 2009-04-29 16:55:27,267 [main] WARN
> >>>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> >>>> Failed to create HOD configuration directory - /tmp/
> >>>>
> PigHod.domU-12-31-38-00-C4-31.dotspots.3204342261033613Retrying...
> >>>> 2009-04-29 16:55:27,312 [main] WARN
> >>>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> >>>> Failed to create HOD configuration directory - /tmp/
> >>>>
> PigHod.domU-12-31-38-00-C4-31.dotspots.3204342307923613Retrying...
> >>>> 2009-04-29 16:55:27,357 [main] WARN
> >>>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> >>>> Failed to create HOD configuration directory - /tmp/
> >>>>
> PigHod.domU-12-31-38-00-C4-31.dotspots.3204342352914613Retrying...
> >>>>
> >>>> More info:
> >>>> I am using the hadoop19.jar from the jira in my pig/lib dir.
> >>>>
> >>>> /////////////////Environment:
> >>>> export HADOOP_HOME=/mnt/dist/app/hadoop-0.19.1
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB