Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Error putting files in the HDFS


Copy link to this message
-
Re: Error putting files in the HDFS
You are welcome Basu.

Not a problem. You can use *bin/hadoop fs -lsr /* to list down all the HDFS
files and directories. See which files are no longer required and delete
them using *bin/hadoop fs -rm /path/to/the/file*

Warm Regards,
Tariq
cloudfront.blogspot.com
On Tue, Oct 8, 2013 at 11:59 PM, Basu,Indrashish <[EMAIL PROTECTED]> wrote:

> **
>
>
>
> Hi Tariq,
>
> Thanks a lot for your help.
>
> Can you please let me know the path where I can check the old files in the
> HDFS and remove them accordingly. I am sorry to bother with these
> questions, I am absolutely new to Hadoop.
>
> Thanks again for your time and pateince.
>
>
>
> Regards,
>
> Indrashish
>
>
>
>
>
> On Tue, 8 Oct 2013 23:51:30 +0530, Mohammad Tariq wrote:
>
> You don't have any more space left in your HDFS. Delete some old data or
> add additional storage.
>
>  Warm Regards,
> Tariq
> cloudfront.blogspot.com
>
>
> On Tue, Oct 8, 2013 at 11:47 PM, Basu,Indrashish <[EMAIL PROTECTED]>wrote:
>
>>
>>
>> Hi ,
>>
>> Just to update on this, I have deleted all the old logs and files from
>> the /tmp and /app/hadoop directory, and restarted all the nodes, I have now
>> 1 datanode available as per the below information :
>>
>> Configured Capacity: 3665985536 (3.41 GB)
>> Present Capacity: 24576 (24 KB)
>>
>> DFS Remaining: 0 (0 KB)
>> DFS Used: 24576 (24 KB)
>> DFS Used%: 100%
>>
>> Under replicated blocks: 0
>> Blocks with corrupt replicas: 0
>> Missing blocks: 0
>>
>> -------------------------------------------------
>> Datanodes available: 1 (1 total, 0 dead)
>>
>> Name: 10.227.56.195:50010
>> Decommission Status : Normal
>> Configured Capacity: 3665985536 (3.41 GB)
>> DFS Used: 24576 (24 KB)
>> Non DFS Used: 3665960960 (3.41 GB)
>> DFS Remaining: 0(0 KB)
>> DFS Used%: 0%
>> DFS Remaining%: 0%
>> Last contact: Tue Oct 08 11:12:19 PDT 2013
>>
>>
>> However when I tried putting the files back in HDFS, I am getting the
>> same error as stated earlier. Do I need to clear some space for the HDFS ?
>>
>> Regards,
>> Indrashish
>>
>>
>>
>> On Tue, 08 Oct 2013 14:01:19 -0400, Basu,Indrashish wrote:
>>
>>> Hi Jitendra,
>>>
>>> This is what I am getting in the datanode logs :
>>>
>>> 2013-10-07 11:27:41,960 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage: Storage directory
>>> /app/hadoop/tmp/dfs/data is not formatted.
>>> 2013-10-07 11:27:41,961 INFO
>>> org.apache.hadoop.hdfs.server.common.Storage: Formatting ...
>>> 2013-10-07 11:27:42,094 INFO
>>> org.apache.hadoop.hdfs.server.datanode.DataNode: Registered
>>> FSDatasetStatusMBean
>>> 2013-10-07 11:27:42,099 INFO
>>> org.apache.hadoop.hdfs.server.datanode.DataNode: Opened info server at
>>> 50010
>>> 2013-10-07 11:27:42,107 INFO
>>> org.apache.hadoop.hdfs.server.datanode.DataNode: Balancing bandwith is
>>> 1048576 bytes/s
>>> 2013-10-07 11:27:42,369 INFO org.mortbay.log: Logging to
>>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
>>> org.mortbay.log.Slf4jLog
>>> 2013-10-07 11:27:42,632 INFO org.apache.hadoop.http.HttpServer: Port
>>> returned by webServer.getConnectors()[0].getLocalPort() before open()
>>> is -1. Opening the listener on 50075
>>> 2013-10-07 11:27:42,633 INFO org.apache.hadoop.http.HttpServer:
>>> listener.getLocalPort() returned 50075
>>> webServer.getConnectors()[0].getLocalPort() returned 50075
>>> 2013-10-07 11:27:42,634 INFO org.apache.hadoop.http.HttpServer: Jetty
>>> bound to port 50075
>>> 2013-10-07 11:27:42,634 INFO org.mortbay.log: jetty-6.1.14
>>> 2013-10-07 11:31:29,821 INFO org.mortbay.log: Started
>>> SelectChannelConnector@0.0.0.0:50075
>>> 2013-10-07 11:31:29,843 INFO
>>> org.apache.hadoop.metrics.jvm.JvmMetrics: Initializing JVM Metrics
>>> with processName=DataNode, sessionId=null
>>> 2013-10-07 11:31:29,912 INFO
>>> org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing RPC Metrics
>>> with hostName=DataNode, port=50020
>>> 2013-10-07 11:31:29,922 INFO org.apache.hadoop.ipc.Server: IPC Server
>>> Responder: starting
>>> 2013-10-07 11:31:29,922 INFO org.apache.hadoop.ipc.Server: IPC Server
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB