+
Vikas Jadhav 2013-01-16, 07:46
+
Harsh J 2013-01-16, 07:53
+
Mohammad Tariq 2013-01-16, 13:03
-
Re: Hadoop NON DFS spaceJean-Marc Spaggiari 2013-01-17, 13:55
Hi Vikas,
You might want to check your logs. MR can generate huge logs depending
on what you are logging, and they are not on the DFS. The are on
non-dfs. If it's comming from there, you can change the loglevel to
reduce the size of the output.
On my own cluster, I turned the logs to debug and forgot to remove
that. At the end I ran into disks space issues because of that.
HTH
JM
2013/1/17, Vikas Jadhav <[EMAIL PROTECTED]>:
> Here is my problem
> I am using bulk loading for Hbase using MapReduce Program
>
> Configured Capacity : 15.5 GB DFS Used : 781.91 MB Non DFS Used : 1.68
> GBDFS Remaining:13.06 GBDFS Used%:4.93 %DFS Remaining%:84.26 %
>
> But when i run my program
>
> Configured Capacity : 15.5 GB DFS Used : 819.69 MB Non DFS Used :
> 14.59 GBDFS Remaining:116.01 MBDFS Used%:5.16 %DFS Remaining%:0.73 %
>
> I have disable WAL in hbase still its consuming non-dfs
> and my program fails have tried lot times but no luck
>
> SO WHAT SHLOULD I DO SO THAT NON DFS WILL NOT CONSUME WHOLE SPACE
>
> I AM ALSO NOT ABLE TO FIND REASON BEHIND usage of non-dfs space to this
> large extent
>
>
> 13/01/17 08:44:07 INFO mapred.JobClient: map 83% reduce 22%
> 13/01/17 08:44:09 INFO mapred.JobClient: map 84% reduce 22%
> 13/01/17 08:44:12 INFO mapred.JobClient: map 85% reduce 22%
> 13/01/17 08:44:15 INFO mapred.JobClient: map 86% reduce 22%
> 13/01/17 08:44:18 INFO mapred.JobClient: map 87% reduce 22%
> 13/01/17 08:44:22 INFO mapred.JobClient: map 79% reduce 22%
> 13/01/17 08:44:25 INFO mapred.JobClient: map 80% reduce 25%
> 13/01/17 08:44:27 INFO mapred.JobClient: Task Id :
> attempt_201301170837_0004_m_000009_0, Status : FAILED
> FSError: java.io.IOException: No space left on device
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Creation of
> /tmp/hadoop-cfgsas1/mapred/local/userlogs/job_201301170837_0004/attempt_201301170837_0004_m_000009_0.cleanup
> failed.
> at
> org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:104)
> at
> org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
> at
> org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)
> 13/01/17 08:44:27 WARN mapred.JobClient: Error reading task outputhttp://
> rdcesx12078.race.sas.com:50060/tasklog?plaintext=true&attemptid=attempt_201301170837_0004_m_000009_0&filter=stdout
> 13/01/17 08:44:27 WARN mapred.JobClient: Error reading task outputhttp://
> rdcesx12078.race.sas.com:50060/tasklog?plaintext=true&attemptid=attempt_201301170837_0004_m_000009_0&filter=stderr
> 13/01/17 08:44:28 INFO mapred.JobClient: map 82% reduce 25%
> 13/01/17 08:44:31 INFO mapred.JobClient: map 83% reduce 25%
> 13/01/17 08:45:07 INFO mapred.JobClient: map 83% reduce 27%
>
>
>
>
> On Wed, Jan 16, 2013 at 6:43 PM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> I think you can still run with the OS on another drive, or on a live
>> USB drive, or even on the memory only, loaded from the network while
>> the server is booting from the network drive, etc. No?
>>
>> JM
>>
>> 2013/1/16, Mohammad Tariq <[EMAIL PROTECTED]>:
>> > That would be really cool Chris.
>> > +1 for that.
>> >
>> > Warm Regards,
>> > Tariq
>> > https://mtariq.jux.com/
>> > cloudfront.blogspot.com
>> >
>> >
>> > On Wed, Jan 16, 2013 at 6:15 PM, Chris Embree <[EMAIL PROTECTED]>
>> > wrote:
>> >
>> >> Ha, you joke, but we're planning on running with no local OS. If it
>> >> works
>> >> as planned I'll post a nice summary of our approach. :)
>> >>
>> >>
>> >> On Wed, Jan 16, 2013 at 2:53 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>> >>
>> >>> <kidding> Wipe your OS out. </kidding>
>> >>>
>> >>> Please read: http://search-hadoop.com/m/9Qwi9UgMOe
>> >>>
>> >>>
>> >>> On Wed, Jan 16, 2013 at 1:16 PM, Vikas Jadhav
>> >>> <[EMAIL PROTECTED]>wrote:
You might want to check your logs. MR can generate huge logs depending
on what you are logging, and they are not on the DFS. The are on
non-dfs. If it's comming from there, you can change the loglevel to
reduce the size of the output.
On my own cluster, I turned the logs to debug and forgot to remove
that. At the end I ran into disks space issues because of that.
HTH
JM
2013/1/17, Vikas Jadhav <[EMAIL PROTECTED]>:
> Here is my problem
> I am using bulk loading for Hbase using MapReduce Program
>
> Configured Capacity : 15.5 GB DFS Used : 781.91 MB Non DFS Used : 1.68
> GBDFS Remaining:13.06 GBDFS Used%:4.93 %DFS Remaining%:84.26 %
>
> But when i run my program
>
> Configured Capacity : 15.5 GB DFS Used : 819.69 MB Non DFS Used :
> 14.59 GBDFS Remaining:116.01 MBDFS Used%:5.16 %DFS Remaining%:0.73 %
>
> I have disable WAL in hbase still its consuming non-dfs
> and my program fails have tried lot times but no luck
>
> SO WHAT SHLOULD I DO SO THAT NON DFS WILL NOT CONSUME WHOLE SPACE
>
> I AM ALSO NOT ABLE TO FIND REASON BEHIND usage of non-dfs space to this
> large extent
>
>
> 13/01/17 08:44:07 INFO mapred.JobClient: map 83% reduce 22%
> 13/01/17 08:44:09 INFO mapred.JobClient: map 84% reduce 22%
> 13/01/17 08:44:12 INFO mapred.JobClient: map 85% reduce 22%
> 13/01/17 08:44:15 INFO mapred.JobClient: map 86% reduce 22%
> 13/01/17 08:44:18 INFO mapred.JobClient: map 87% reduce 22%
> 13/01/17 08:44:22 INFO mapred.JobClient: map 79% reduce 22%
> 13/01/17 08:44:25 INFO mapred.JobClient: map 80% reduce 25%
> 13/01/17 08:44:27 INFO mapred.JobClient: Task Id :
> attempt_201301170837_0004_m_000009_0, Status : FAILED
> FSError: java.io.IOException: No space left on device
> java.lang.Throwable: Child Error
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:271)
> Caused by: java.io.IOException: Creation of
> /tmp/hadoop-cfgsas1/mapred/local/userlogs/job_201301170837_0004/attempt_201301170837_0004_m_000009_0.cleanup
> failed.
> at
> org.apache.hadoop.mapred.TaskLog.createTaskAttemptLogDir(TaskLog.java:104)
> at
> org.apache.hadoop.mapred.DefaultTaskController.createLogDir(DefaultTaskController.java:71)
> at
> org.apache.hadoop.mapred.TaskRunner.prepareLogFiles(TaskRunner.java:316)
> at org.apache.hadoop.mapred.TaskRunner.run(TaskRunner.java:228)
> 13/01/17 08:44:27 WARN mapred.JobClient: Error reading task outputhttp://
> rdcesx12078.race.sas.com:50060/tasklog?plaintext=true&attemptid=attempt_201301170837_0004_m_000009_0&filter=stdout
> 13/01/17 08:44:27 WARN mapred.JobClient: Error reading task outputhttp://
> rdcesx12078.race.sas.com:50060/tasklog?plaintext=true&attemptid=attempt_201301170837_0004_m_000009_0&filter=stderr
> 13/01/17 08:44:28 INFO mapred.JobClient: map 82% reduce 25%
> 13/01/17 08:44:31 INFO mapred.JobClient: map 83% reduce 25%
> 13/01/17 08:45:07 INFO mapred.JobClient: map 83% reduce 27%
>
>
>
>
> On Wed, Jan 16, 2013 at 6:43 PM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> I think you can still run with the OS on another drive, or on a live
>> USB drive, or even on the memory only, loaded from the network while
>> the server is booting from the network drive, etc. No?
>>
>> JM
>>
>> 2013/1/16, Mohammad Tariq <[EMAIL PROTECTED]>:
>> > That would be really cool Chris.
>> > +1 for that.
>> >
>> > Warm Regards,
>> > Tariq
>> > https://mtariq.jux.com/
>> > cloudfront.blogspot.com
>> >
>> >
>> > On Wed, Jan 16, 2013 at 6:15 PM, Chris Embree <[EMAIL PROTECTED]>
>> > wrote:
>> >
>> >> Ha, you joke, but we're planning on running with no local OS. If it
>> >> works
>> >> as planned I'll post a nice summary of our approach. :)
>> >>
>> >>
>> >> On Wed, Jan 16, 2013 at 2:53 AM, Harsh J <[EMAIL PROTECTED]> wrote:
>> >>
>> >>> <kidding> Wipe your OS out. </kidding>
>> >>>
>> >>> Please read: http://search-hadoop.com/m/9Qwi9UgMOe
>> >>>
>> >>>
>> >>> On Wed, Jan 16, 2013 at 1:16 PM, Vikas Jadhav
>> >>> <[EMAIL PROTECTED]>wrote:
+
Harsh J 2013-01-17, 14:07
+
Vikas Jadhav 2013-01-17, 14:11
