When I start a job to read data from HDFS I start getting these errors.
Does anyone know what this means and how to resolve it?

2012-03-15 10:41:31,402 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.204:50010java.io.EOFException
2012-03-15 10:41:31,402 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_-6402969611996946639_11837
2012-03-15 10:41:31,403 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Excluding datanode 164.28.62.204:50010
2012-03-15 10:41:31,406 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.198:50010java.io.EOFException
2012-03-15 10:41:31,406 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_-5442664108986165368_11838
2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.197:50010java.io.EOFException
2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_-3373089616877234160_11838
2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Excluding datanode 164.28.62.198:50010
2012-03-15 10:41:31,409 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Excluding datanode 164.28.62.197:50010
2012-03-15 10:41:31,410 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.204:50010java.io.EOFException
2012-03-15 10:41:31,410 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_4481292025401332278_11838
2012-03-15 10:41:31,411 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Excluding datanode 164.28.62.204:50010
2012-03-15 10:41:31,412 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.200:50010java.io.EOFException
2012-03-15 10:41:31,412 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_-5326771177080888701_11838
2012-03-15 10:41:31,413 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Excluding datanode 164.28.62.200:50010
2012-03-15 10:41:31,414 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.197:50010java.io.EOFException
2012-03-15 10:41:31,414 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_-8073750683705518772_11839
2012-03-15 10:41:31,415 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Excluding datanode 164.28.62.197:50010
2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.199:50010java.io.EOFException
2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Exception in createBlockOutputStream 164.28.62.198:50010java.io.EOFException
2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_441003866688859169_11838
2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Abandoning block blk_-466858474055876377_11839
2012-03-15 10:41:31,417 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
Excluding datanode 164.28.62.198:50010
2012-03-15 10:41:31,417 [Thread-5] WARN  org.apache.hadoop.hdfs.DFSClient -
DataStreamer Exception: java.io.IOException: Unable to create new block.
        at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.nextBlockOutputStream(DFSClient.java:3446)
        at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream.access$2100(DFSClient.java:2627)
        at
org.apache.hadoop.hdfs.DFSClient$DFSOutputStream$DataStreamer.run(DFSClient.java:2822)

On 03/15/2012 03:06 PM, Mohit Anchlia wrote:
> When I start a job to read data from HDFS I start getting these errors.
> Does anyone know what this means and how to resolve it?
>
> 2012-03-15 10:41:31,402 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.204:50010java.io.EOFException
> 2012-03-15 10:41:31,402 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_-6402969611996946639_11837
> 2012-03-15 10:41:31,403 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Excluding datanode 164.28.62.204:50010
> 2012-03-15 10:41:31,406 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.198:50010java.io.EOFException
> 2012-03-15 10:41:31,406 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_-5442664108986165368_11838
> 2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.197:50010java.io.EOFException
> 2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_-3373089616877234160_11838
> 2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Excluding datanode 164.28.62.198:50010
> 2012-03-15 10:41:31,409 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Excluding datanode 164.28.62.197:50010
> 2012-03-15 10:41:31,410 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.204:50010java.io.EOFException
> 2012-03-15 10:41:31,410 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_4481292025401332278_11838
> 2012-03-15 10:41:31,411 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Excluding datanode 164.28.62.204:50010
> 2012-03-15 10:41:31,412 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.200:50010java.io.EOFException
> 2012-03-15 10:41:31,412 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_-5326771177080888701_11838
> 2012-03-15 10:41:31,413 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Excluding datanode 164.28.62.200:50010
> 2012-03-15 10:41:31,414 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.197:50010java.io.EOFException
> 2012-03-15 10:41:31,414 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_-8073750683705518772_11839
> 2012-03-15 10:41:31,415 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Excluding datanode 164.28.62.197:50010
> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.199:50010java.io.EOFException
> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Exception in createBlockOutputStream 164.28.62.198:50010java.io.EOFException
> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_441003866688859169_11838
> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Abandoning block blk_-466858474055876377_11839
> 2012-03-15 10:41:31,417 [Thread-5] INFO  org.apache.hadoop.hdfs.DFSClient -
> Excluding datanode 164.28.62.198:50010
> 2012-03-15 10:41:31,417 [Thread-5] WARN  org.apache.hadoop.hdfs.DFSClient -
>    
Try shutting down and  restarting hbase.

This is actually just hadoop job over HDFS. I am assuming you also know why
this is erroring out?

On Thu, Mar 15, 2012 at 1:02 PM, Gopal <[EMAIL PROTECTED]> wrote:

>  On 03/15/2012 03:06 PM, Mohit Anchlia wrote:
>
>> When I start a job to read data from HDFS I start getting these errors.
>> Does anyone know what this means and how to resolve it?
>>
>> 2012-03-15 10:41:31,402 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.204:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,402 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_-6402969611996946639_11837
>> 2012-03-15 10:41:31,403 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Excluding datanode 164.28.62.204:50010
>> 2012-03-15 10:41:31,406 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.198:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,406 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_-5442664108986165368_11838
>> 2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.197:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_-3373089616877234160_11838
>> 2012-03-15 10:41:31,407 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Excluding datanode 164.28.62.198:50010
>> 2012-03-15 10:41:31,409 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Excluding datanode 164.28.62.197:50010
>> 2012-03-15 10:41:31,410 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.204:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,410 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_4481292025401332278_11838
>> 2012-03-15 10:41:31,411 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Excluding datanode 164.28.62.204:50010
>> 2012-03-15 10:41:31,412 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.200:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,412 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_-5326771177080888701_11838
>> 2012-03-15 10:41:31,413 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Excluding datanode 164.28.62.200:50010
>> 2012-03-15 10:41:31,414 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.197:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,414 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_-8073750683705518772_11839
>> 2012-03-15 10:41:31,415 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Excluding datanode 164.28.62.197:50010
>> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.199:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Exception in createBlockOutputStream 164.28.62.198:50010java.io.**
>> EOFException
>> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_441003866688859169_11838
>> 2012-03-15 10:41:31,416 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Abandoning block blk_-466858474055876377_11839
>> 2012-03-15 10:41:31,417 [Thread-5] INFO  org.apache.hadoop.hdfs.**DFSClient
>> -
>> Excluding datanode 164.28.62.198:50010
>> 2012-03-15 10:41:31,417 [Thread-5] WARN  org.apache.hadoop.hdfs.**DFSClient
>> -
>>
>>
> Try shutting down and  restarting hbase.
>

Hi,
 Seems like HDFS is in safemode.

On Fri, Mar 16, 2012 at 1:37 AM, Mohit Anchlia <[EMAIL PROTECTED]>wrote:

> This is actually just hadoop job over HDFS. I am assuming you also know why
> this is erroring out?
>
> On Thu, Mar 15, 2012 at 1:02 PM, Gopal <[EMAIL PROTECTED]> wrote:
>
> >  On 03/15/2012 03:06 PM, Mohit Anchlia wrote:
> >
> >> When I start a job to read data from HDFS I start getting these errors.
> >> Does anyone know what this means and how to resolve it?
> >>
> >> 2012-03-15 10:41:31,402 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,402 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_-6402969611996946639_11837
> >> 2012-03-15 10:41:31,403 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Excluding datanode 164.28.62.204:50010
> >> 2012-03-15 10:41:31,406 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,406 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_-5442664108986165368_11838
> >> 2012-03-15 10:41:31,407 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,407 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_-3373089616877234160_11838
> >> 2012-03-15 10:41:31,407 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Excluding datanode 164.28.62.198:50010
> >> 2012-03-15 10:41:31,409 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Excluding datanode 164.28.62.197:50010
> >> 2012-03-15 10:41:31,410 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,410 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_4481292025401332278_11838
> >> 2012-03-15 10:41:31,411 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Excluding datanode 164.28.62.204:50010
> >> 2012-03-15 10:41:31,412 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.200:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,412 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_-5326771177080888701_11838
> >> 2012-03-15 10:41:31,413 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Excluding datanode 164.28.62.200:50010
> >> 2012-03-15 10:41:31,414 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,414 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_-8073750683705518772_11839
> >> 2012-03-15 10:41:31,415 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Excluding datanode 164.28.62.197:50010
> >> 2012-03-15 10:41:31,416 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.199:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,416 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.**
> >> EOFException
> >> 2012-03-15 10:41:31,416 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_441003866688859169_11838
> >> 2012-03-15 10:41:31,416 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Abandoning block blk_-466858474055876377_11839
> >> 2012-03-15 10:41:31,417 [Thread-5] INFO
>  org.apache.hadoop.hdfs.**DFSClient
> >> -
> >> Excluding datanode 164.28.62.198:50010

https://github.com/zinnia-phatak-dev/Nectar

I guess I am trying to see how to debug such problems? I don't see enough
info in the logs.
On Mon, Mar 19, 2012 at 12:48 AM, madhu phatak <[EMAIL PROTECTED]> wrote:

> Hi,
>  Seems like HDFS is in safemode.
>
> On Fri, Mar 16, 2012 at 1:37 AM, Mohit Anchlia <[EMAIL PROTECTED]
> >wrote:
>
> > This is actually just hadoop job over HDFS. I am assuming you also know
> why
> > this is erroring out?
> >
> > On Thu, Mar 15, 2012 at 1:02 PM, Gopal <[EMAIL PROTECTED]> wrote:
> >
> > >  On 03/15/2012 03:06 PM, Mohit Anchlia wrote:
> > >
> > >> When I start a job to read data from HDFS I start getting these
> errors.
> > >> Does anyone know what this means and how to resolve it?
> > >>
> > >> 2012-03-15 10:41:31,402 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,402 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Abandoning block blk_-6402969611996946639_11837
> > >> 2012-03-15 10:41:31,403 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Excluding datanode 164.28.62.204:50010
> > >> 2012-03-15 10:41:31,406 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,406 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Abandoning block blk_-5442664108986165368_11838
> > >> 2012-03-15 10:41:31,407 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,407 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Abandoning block blk_-3373089616877234160_11838
> > >> 2012-03-15 10:41:31,407 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Excluding datanode 164.28.62.198:50010
> > >> 2012-03-15 10:41:31,409 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Excluding datanode 164.28.62.197:50010
> > >> 2012-03-15 10:41:31,410 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.204:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,410 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Abandoning block blk_4481292025401332278_11838
> > >> 2012-03-15 10:41:31,411 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Excluding datanode 164.28.62.204:50010
> > >> 2012-03-15 10:41:31,412 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.200:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,412 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Abandoning block blk_-5326771177080888701_11838
> > >> 2012-03-15 10:41:31,413 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Excluding datanode 164.28.62.200:50010
> > >> 2012-03-15 10:41:31,414 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.197:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,414 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Abandoning block blk_-8073750683705518772_11839
> > >> 2012-03-15 10:41:31,415 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Excluding datanode 164.28.62.197:50010
> > >> 2012-03-15 10:41:31,416 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.199:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,416 [Thread-5] INFO
> >  org.apache.hadoop.hdfs.**DFSClient
> > >> -
> > >> Exception in createBlockOutputStream 164.28.62.198:50010java.io.**
> > >> EOFException
> > >> 2012-03-15 10:41:31,416 [Thread-5] INFO

I have been running several MapReduce jobs on some input text files. They
were working fine earlier and then I suddenly started getting EOFException
every time. Even the jobs that ran fine before (on the exact same input
files) aren't running now. I am a bit perplexed as to what is causing this
error. Here is the error:

12/04/30 12:55:55 INFO mapred.JobClient: Task Id :
attempt_201202240659_6328_m_000001_1, Status : FAILED
java.lang.RuntimeException: java.io.EOFException
        at
org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:128)
        at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:967)
        at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30)
        at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83)
        at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
        at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1253)
        at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1154)
        at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:268)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1115)
        at org.apache.hadoop.mapred.Child.main(Child.java:262)
Caused by: java.io.EOFException
        at java.io.DataInputStream.readInt(DataInputStream.java:375)
        at com.xerox.twitter.bin.UserTime.readFields(UserTime.java:31)
        at
org.apache.hadoop.io.WritableComparator.compare(WritableComparator.java:122)

Since the compare function seems to be involved, here is my custom key
class. Note: I did not include year in the key because all keys have the
same year.

public class UserTime implements WritableComparable<UserTime> {

int id, month, day, year, hour, min, sec;
 public UserTime() {

}
 public UserTime(int u, int mon, int d, int y, int h, int m, int s) {
id = u;
month = mon;
day = d;
year = y;
hour = h;
min = m;
sec = s;
}
 @Override
public void readFields(DataInput in) throws IOException {
// TODO Auto-generated method stub
id = in.readInt();
month = in.readInt();
day = in.readInt();
year = in.readInt();
hour = in.readInt();
min = in.readInt();
sec = in.readInt();
}

@Override
public void write(DataOutput out) throws IOException {
// TODO Auto-generated method stub
out.write(id);
out.write(month);
out.write(day);
out.write(year);
out.write(hour);
out.write(min);
out.write(sec);
}

@Override
public int compareTo(UserTime that) {
// TODO Auto-generated method stub
if(compareUser(that) == 0)
return (compareTime(that));
else if(compareUser(that) == 1)
return 1;
else return -1;
}
 private int compareUser(UserTime that) {
if(id > that.id)
return 1;
else if(id == that.id)
return 0;
else return -1;
}
 //assumes all are from the same year
private int compareTime(UserTime that) {
if(month > that.month ||
(month == that.month && day > that.day) ||
(month == that.month && day == that.day && hour > that.hour) ||
(month == that.month && day == that.day && hour == that.hour && min >
that.min) ||
(month == that.month && day == that.day && hour == that.hour && min =that.min && sec > that.sec))
return 1;
else if(month == that.month && day == that.day && hour == that.hour && min
== that.min && sec == that.sec)
return 0;
else return -1;
}
 public String toString() {
String h, m, s;
if(hour < 10)
h = "0"+hour;
else
h = Integer.toString(hour);
if(min < 10)
m = "0"+min;
else
m = Integer.toString(hour);
if(sec < 10)
s = "0"+min;
else
s = Integer.toString(hour);
return (id+"\t"+month+"/"+day+"/"+year+"\t"+h+":"+m+":"+s);
}
}

Thanks for any help.

Regards,
Keith