|
|
-
MapReduce vs hosts (Cannot resolve the host name)
Jean-Marc Spaggiari 2012-10-11, 13:17
Hi,
I'm facing a small issue, most probably configuration related, that I'm not able to solve.
I'm trying to run the rowcounter.
Here is the command line: export HADOOP_HOME=/home/hadoop/hadoop-1.0.3/; export HBASE_HOME=/home/hbase/hbase-0.94.0/; HADOOP_CLASSPATH=`${HBASE_HOME}/bin/hbase classpath` ${HADOOP_HOME}/bin/hadoop jar ${HBASE_HOME}/hbase-0.94.0.jar rowcounter work_proposed -Dhbase.client.scanner.cachi=10
I'm getting errors for each of my regions: 12/10/11 09:10:45 ERROR mapreduce.TableInputFormatBase: Cannot resolve the host name for /192.168.23.9 because of javax.naming.NameNotFoundException: DNS name not found [response code 3]; remaining name '9.23.168.192.in-addr.arpa'
I have 25 regions for this table, so I have 25 lines like that.
And then it's blocking at: 12/10/11 09:10:45 INFO mapred.JobClient: Running job: job_201209151131_0005 12/10/11 09:10:46 INFO mapred.JobClient: map 0% reduce 0%
Here is an extract of my host file: 192.168.23.9 node5 192.168.23.8 node4 192.168.23.7 node3 192.168.23.6 node2 192.168.23.5 node1
Hadoop is working fine. HBase is working fine.
If I try the shell and do a count, it's working fine.
If I try with bin/hbase org.apache.hadoop.hbase.mapreduce.RowCounter work_proposed it's working fine, but it's doing the regions one by one, and not in parallel. So it's very slow.
I can ping the nodes, by name or IPs, I can ssh them too.
Any idea where I can start to look at?
Thanks,
JM
+
Jean-Marc Spaggiari 2012-10-11, 13:17
-
Re: MapReduce vs hosts (Cannot resolve the host name)
Stack 2012-10-11, 17:20
On Thu, Oct 11, 2012 at 6:17 AM, Jean-Marc Spaggiari <[EMAIL PROTECTED]> wrote: > Any idea where I can start to look at? >
Sounds like DNS -- forward and reverse lookups -- work on the machine you are launching your job from but not out on your cluster. Check DNS on the cluster members? St.Ack
+
Stack 2012-10-11, 17:20
-
Re: MapReduce vs hosts (Cannot resolve the host name)
Jean-Marc Spaggiari 2012-10-11, 17:29
Hi St.Ack,
Thanks for your reply.
Finally, seems that this is not blocking the application. It's stated as "ERROR"s but it's still running after that. After some time, I'm getting some results.
I don't have any DNS on my cluster. All servers have the same host file and static IPs. Is it mandatory to have a DNS to run HBase? Or can it run without?
JM
2012/10/11, Stack <[EMAIL PROTECTED]>: > On Thu, Oct 11, 2012 at 6:17 AM, Jean-Marc Spaggiari > <[EMAIL PROTECTED]> wrote: >> Any idea where I can start to look at? >> > > Sounds like DNS -- forward and reverse lookups -- work on the machine > you are launching your job from but not out on your cluster. Check > DNS on the cluster members? > St.Ack >
+
Jean-Marc Spaggiari 2012-10-11, 17:29
|
|