Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> EBADF: Bad file descriptor


Copy link to this message
-
Re: EBADF: Bad file descriptor
To clarify a little bit, the readahead pool can sometimes spit out this
message if you close a file while a readahead request is in flight.  It's
not an error and just reflects the fact that the file was closed hastily,
probably because of some other bug which is the real problem.

Colin
On Wed, Jul 10, 2013 at 11:50 AM, Colin McCabe <[EMAIL PROTECTED]>wrote:

> That's just a warning message.  It's not causing your problem-- it's just
> a symptom.
>
> You will have to find out why the MR job failed.
>
> best,
> Colin
>
>
> On Wed, Jul 10, 2013 at 8:19 AM, Sanjay Subramanian <
> [EMAIL PROTECTED]> wrote:
>
>>  2013-07-10 07:11:50,131 WARN [Readahead Thread #1]
>> org.apache.hadoop.io.ReadaheadPool: Failed readahead on ifile
>> EBADF: Bad file descriptor
>> at org.apache.hadoop.io.nativeio.NativeIO.posix_fadvise(Native Method)
>> at
>> org.apache.hadoop.io.nativeio.NativeIO.posixFadviseIfPossible(NativeIO.java:145)
>> at
>> org.apache.hadoop.io.ReadaheadPool$ReadaheadRequestImpl.run(ReadaheadPool.java:205)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> at java.lang.Thread.run(Thread.java:662)
>>
>>  Hi
>>
>>  I have a Oozie workflow that runs a MR job and I have started getting
>> this error past two days in one of the MR jobs that is being processed.
>> However if I run it again , it succeeds :-(  but about 1 hr is wasted in
>> the process.
>>
>>  Any clues ?
>>
>>  Or should I post this issue in the Oozie postings ?
>>
>>  Thanks
>>
>>  sanjay
>>
>>    Configuration   Name Value   impression.log.record.cached.tag cached>> impression.log.record.end.tag [end
>> impressions.mapreduce.conf.file.full.path
>> /workflows/impressions/config/aggregations.conf<http://thv01:8888/filebrowser/view/workflows/impressions/config/aggregations.conf>
>> mapred.job.queue.name default  mapred.mapper.new-api true
>> mapred.reducer.new-api true  mapreduce.input.fileinputformat.inputdir
>> /data/input/impressionlogs/outpdirlogs/9999-99-99<http://thv01:8888/filebrowser/view/data/input/impressionlogs/outpdirlogs/9999-99-99>
>> mapreduce.job.inputformat.class
>> com.wizecommerce.utils.mapred.ZipMultipleLineRecordInputFormat
>> mapreduce.job.map.class
>> com.wizecommerce.parser.mapred.OutpdirImpressionLogMapper
>> mapreduce.job.maps 500  mapreduce.job.name
>> OutpdirImpressions_0000475-130611151004460-oozie-oozi-W
>> mapreduce.job.output.value.class org.apache.hadoop.io.Text
>> mapreduce.job.outputformat.class
>> com.wizecommerce.utils.mapred.NextagTextOutputFormat
>> mapreduce.job.reduce.class
>> com.wizecommerce.parser.mapred.OutpdirImpressionLogReducer
>> mapreduce.job.reduces 8  mapreduce.map.output.compress true
>> mapreduce.map.output.compress.codec
>> org.apache.hadoop.io.compress.SnappyCodec  mapreduce.map.output.key.class
>> org.apache.hadoop.io.Textorg.apache.hadoop.io.Text
>> mapreduce.map.output.value.class
>> com.wizecommerce.parser.dao.OutpdirLogRecord
>> mapreduce.output.fileoutputformat.compress true
>> mapreduce.output.fileoutputformat.compress.codec
>> com.hadoop.compression.lzo.LzopCodec
>> mapreduce.output.fileoutputformat.outputdir
>> /data/output/impressions/outpdir/9999-99-99/0000475-130611151004460-oozie-oozi-W/outpdir_impressions_ptitle<http://thv01:8888/filebrowser/view/data/output/impressions/outpdir/9999-99-99/0000475-130611151004460-oozie-oozi-W/outpdir_impressions_ptitle>
>> mapreduce.tasktracker.map.tasks.maximum 12
>> mapreduce.tasktracker.reduce.tasks.maximum 8
>> outpdir.log.exclude.processing.datatypes header,sellerhidden
>>
>>
>>
>> CONFIDENTIALITY NOTICE
>> =====================>> This email message and any attachments are for the exclusive use of the
>> intended recipient(s) and may contain confidential and privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB