Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> log


But my files are still there. As an example

-rw-r--r--  10 User hadmin    6392159 2013-02-05 21:32
/var/lib/hadoop-hdfs/cache/mapred/mapred/staging/User/.staging/job_201302050715_0032/job.jar

Thanks,
On Fri, Apr 19, 2013 at 2:51 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:

> This basically happens while running a mapreduce job. When a map reduce
> job is triggered the job files are put in hdfs with high replication (
> replication is controlled by  - 'mapred.submit.replication'  default value
> is 10).
>
> The job files are cleaned up after the job is completed and hence that
> could be the reason you are seeing the hdfs file system status as healthy
> after running the job.
>
>
> On Fri, Apr 19, 2013 at 1:04 PM, Mohit Vadhera <
> [EMAIL PROTECTED]> wrote:
>
>> its one (1). Output is below.
>>
>> .......Status: HEALTHY
>>  Total size:    903709673179 B
>>  Total dirs:    2906
>>  Total files:   11110
>>  Total blocks (validated):      20906 (avg. block size 43227287 B)
>>  Minimally replicated blocks:   20906 (100.0 %)
>>  Over-replicated blocks:        0 (0.0 %)
>>  Under-replicated blocks:       248 (1.1862624 %)
>>  Mis-replicated blocks:         0 (0.0 %)
>>  Default replication factor:    1
>>  Average block replication:     1.0
>>  Corrupt blocks:                0
>>  Missing replicas:              2232 (9.646469 %)
>>  Number of data-nodes:          1
>>  Number of racks:               1
>> FSCK ended at Fri Apr 19 03:47:04 EDT 2013 in 2224 milliseconds
>>
>>
>> The filesystem under path '/' is HEALTHY
>>
>>
>>
>> On Fri, Apr 19, 2013 at 12:28 PM, S, Manoj <[EMAIL PROTECTED]> wrote:
>>
>>>  It means that some of your data blocks are not replicated as intended.
>>> What is the value of “dfs.replication” in your hadoop-site.xml file?****
>>>
>>> ** **
>>>
>>> Can you paste the output of ****
>>>
>>> ** **
>>>
>>> *bin/hadoop fsck /****
>>>
>>> ** **
>>>
>>> --****
>>>
>>> Manoj****
>>>
>>> ** **
>>>
>>> *From:* Mohit Vadhera [mailto:[EMAIL PROTECTED]]
>>> *Sent:* Friday, April 19, 2013 12:09 PM
>>> *To:* <[EMAIL PROTECTED]>
>>> *Subject:* log****
>>>
>>> ** **
>>>
>>> Can anybody let me know the meaning of the below log plz " Target
>>> Replicas is 10 but found 1 replica(s)." ?
>>>
>>> /var/lib/hadoop-hdfs/cache/mapred/mapred/staging/test_user/.staging/job_201302180313_0623/job.split:
>>> Under replicated
>>> BP-2091347308-172.20.3.119-1356632249303:blk_6297333561560198850_70720.
>>> Target
>>> Replicas is 10 but found 1 replica(s).
>>>
>>> ****
>>>
>>> Thanks,****
>>>
>>
>>
>