Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: Hadoop 2.2.0 from source configuration


Copy link to this message
-
Re: Hadoop 2.2.0 from source configuration
Adam Kawa 2013-12-03, 18:55
Daniel,

I see that in previous hdfs report, you had: hosta.subdom1.tld1, but now
you have feynman.cids.ca. What is the content of your /etc/hosts file, and
output of $hostname command?
2013/12/3 Daniel Savard <[EMAIL PROTECTED]>

> I did that more than once, I just retry it from the beginning. I zapped
> the directories and recreated them with hdfs namenode -format and restarted
> HDFS and I am still getting the very same error.
>
> I have posted previously the report. Is there anything in this report that
> indicates I am not having enough free space somewhere? That's the only
> thing I can see may cause this problem after everything I read on the
> subject. I am new to Hadoop and I just want to setup a standalone node for
> starting to experiment a while with it before going ahead with a complete
> cluster.
>
> I repost the report for convenience:
>
> Configured Capacity: 2939899904 (2.74 GB)
> Present Capacity: 534421504 (509.66 MB)
> DFS Remaining: 534417408 (509.66 MB)
>
> DFS Used: 4096 (4 KB)
> DFS Used%: 0.00%
> Under replicated blocks: 0
> Blocks with corrupt replicas: 0
> Missing blocks: 0
>
> -------------------------------------------------
> Datanodes available: 1 (1 total, 0 dead)
>
> Live datanodes:
> Name: 127.0.0.1:50010 (feynman.cids.ca)
> Hostname: feynman.cids.ca
> Decommission Status : Normal
> Configured Capacity: 2939899904 (2.74 GB)
>
> DFS Used: 4096 (4 KB)
> Non DFS Used: 2405478400 (2.24 GB)
> DFS Remaining: 534417408 (509.66 MB)
> DFS Used%: 0.00%
> DFS Remaining%: 18.18%
> Last contact: Tue Dec 03 13:37:02 EST 2013
>
>
> -----------------
> Daniel Savard
>
>
> 2013/12/3 Adam Kawa <[EMAIL PROTECTED]>
>
>> Daniel,
>>
>> It looks that you can only communicate with NameNode to do
>> "metadata-only" operations (e.g. listing, creating a dir, empty file)...
>>
>> Did you format the NameNode correctly?
>> A quite similar issue is described here:
>> http://www.manning-sandbox.com/thread.jspa?messageID=126741. The last
>> reply says: "The most common is that you have reformatted the namenode
>> leaving it in an inconsistent state. The most common solution is to stop
>> dfs, remove the contents of the dfs directories on all the machines, run
>> “hadoop namenode -format” on the controller, then restart dfs. That
>> consistently fixes the problem for me. This may be serious overkill but it
>> works."
>>
>>
>> 2013/12/3 Daniel Savard <[EMAIL PROTECTED]>
>>
>>> Thanks Arun,
>>>
>>> I already read and did everything recommended at the referred URL. There
>>> isn't any error message in the logfiles. The only error message appears
>>> when I try to put a non-zero file on the HDFS as posted above. Beside that,
>>> absolutely nothing in the logs is telling me something is wrong with the
>>> configuration so far.
>>>
>>> Is there some sort of diagnostic tool that can query/ping each server to
>>> make sure it responds properly to requests? When trying to put my file, in
>>> the datanode log I see nothing, the message appears in the namenode log. Is
>>> this the expected behavior or should I see at least some kind of request
>>> message in the datanode logfile?
>>>
>>>
>>> -----------------
>>> Daniel Savard
>>>
>>>
>>> 2013/12/2 Arun C Murthy <[EMAIL PROTECTED]>
>>>
>>>> Daniel,
>>>>
>>>>  Apologies if you had a bad experience. If you can point them out to
>>>> us, we'd be more than happy to fix it - alternately, we'd *love* it if you
>>>> could help us improve docs too.
>>>>
>>>>  Now, for the problem at hand:
>>>> http://wiki.apache.org/hadoop/CouldOnlyBeReplicatedTo is one place to
>>>> look. Basically NN cannot find any datanodes. Anything in your NN logs to
>>>> indicate trouble?
>>>>
>>>>  Also, pls feel free to open liras with issues you find and we'll help.
>>>>
>>>> thanks,
>>>> Arun
>>>>
>>>> On Dec 2, 2013, at 8:44 AM, Daniel Savard <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>> André,
>>>>
>>>> good for you that greedy instructions on the reference page were enough
>>>> to setup your cluster. However, read them again and see how many