Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Pig accessing remote hadoop cluster


Copy link to this message
-
Re: Pig accessing remote hadoop cluster
Are you able to connect on tcp/54311 to that machine?  The higher-numbered
ports used by Hadoop/HBase are often blocked by firewalls.

Try using netcat -- "nc NAMENODEHOST 54311".  Success would be indicated by
a hanging connection that's waiting for input.

Norbert

On Wed, Mar 28, 2012 at 1:48 PM, Iman E <[EMAIL PROTECTED]> wrote:

> Yes, I am able to ping the node with the jobtracker and namenode running
> on it.
>
>
> ________________________________
>  From: Prashant Kommireddi <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Wednesday, March 28, 2012 1:24 PM
> Subject: Re: Pig accessing remote hadoop cluster
>
> Are you able to ping Namenode from your localhost?
>
> -Prashant
>
> On Mar 28, 2012, at 10:19 AM, Iman E <[EMAIL PROTECTED]> wrote:
>
> > Hello all,
> > I used to run pig on the same node where the hadoop job tracker is
> running and everything was fine. Now I am trying to run pig on my laptop to
> access the cluster where hadoop is running but this alternative fails.
> > I am running pig 0.8. I have copied the hadoop configuration directory
> to my local machine and have pointed pig to use its configuration files.
> Yet, pig fails to establish the connection with the remote hadoop
> jobtracker. Any suggestions of what should i do to fix this error and get
> it connect to the remote jobtracker?
> >
> >
> > The error returned is as follows:
> >
> > 2012-03-28 10:00:25,614 [main] INFO  org.apache.pig.Main - Logging error
> messages to:
> > /home/xxx/pig_1332943225604.log
> > 2012-03-28 10:00:25,809 [main] INFO
> > org.apache.pig.backend.hadoop.executionengine.HExecutionEngine -
> > Connecting to hadoop file system at: hdfs://XXX.XXX.XXX.XXX:54311
> > 2012-03-28
> > 10:00:46,922 [main] INFO  org.apache.hadoop.ipc.Client - Retrying
> > connect to server: /XXX.XXX.XXX.XXX :54311. Already tried 0 time(s).
> > 2012-03-28
> > 10:01:07,935 [main] INFO  org.apache.hadoop.ipc.Client - Retrying
> > connect to server: /XXX.XXX.XXX.XXX :54311. Already tried 1 time(s).
> >
> >
> >
> > The error appearing in the log files is as follows:
> >
> > Error before Pig is launched
> > ----------------------------
> > ERROR
> > 2999: Unexpected internal error. Failed to create DataStorage
> >
> > java.lang.RuntimeException:
> > Failed to create DataStorage
> >     at
> >
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.init(HDataStorage.java:75)
> >
> > at
> >
> org.apache.pig.backend.hadoop.datastorage.HDataStorage.<init>(HDataStorage.java:58)
> >
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:213)
> >
> > at
> >
> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.init(HExecutionEngine.java:133)
> >
> > at org.apache.pig.impl.PigContext.connect(PigContext.java:183)
> >
> > at
> > org.apache.pig.PigServer.<init>(PigServer.java:233)
> >     at
> > org.apache.pig.PigServer.<init>(PigServer.java:222)
> >     at
> > org.apache.pig.tools.grunt.Grunt.<init>(Grunt.java:55)
> >     at
> > org.apache.pig.Main.run(Main.java:462)
> >     at
> > org.apache.pig.Main.main(Main.java:107)
> > Caused by:
> > java.net.SocketTimeoutException: Call to /XXX.XXX.XXX.XXX :54311 failed
> on
> > socket timeout exception: java.net.SocketTimeoutException: 20000 millis
> > timeout while waiting for channel to be ready for connect. ch :
> > java.nio.channels.SocketChannel[connection-pending
> > remote=/XXX.XXX.XXX.XXX :54311]
> >     at
> > org.apache.hadoop.ipc.Client.wrapException(Client.java:771)
> >     at
> > org.apache.hadoop.ipc.Client.call(Client.java:743)
> >     at
> > org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
> >     at
> > $Proxy0.getProtocolVersion(Unknown
> > Source)
> >     at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
> >
> > at
> > org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
> >
> > at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB