Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> IOException: too many length or distance symbols


Copy link to this message
-
Re: IOException: too many length or distance symbols
Good to know Prashant, thanks for getting back!

On Mon, Jul 30, 2012 at 12:54 AM, Prashant Kommireddi
<[EMAIL PROTECTED]> wrote:
> Thanks Harsh.
>
> On digging some more it appears there was a data corruption issue with
> the file that caused the exception. After having regenerated the gzip
> file from source I no longer see the issue.
>
>
> On Jul 20, 2012, at 8:48 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> Prashant,
>>
>> Can you add in some context on how these files were written, etc.?
>> Perhaps open a JIRA with a sample file and test-case to reproduce
>> this? Other env stuff with info on version of hadoop, etc. would help
>> too.
>>
>> On Sat, Jul 21, 2012 at 2:05 AM, Prashant Kommireddi
>> <[EMAIL PROTECTED]> wrote:
>>> I am seeing these exceptions, anyone know what they might be caused due to?
>>> Case of corrupt file?
>>>
>>> java.io.IOException: too many length or distance symbols
>>>        at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.inflateBytesDirect(Native
>>> Method)
>>>        at org.apache.hadoop.io.compress.zlib.ZlibDecompressor.decompress(ZlibDecompressor.java:221)
>>>        at org.apache.hadoop.io.compress.DecompressorStream.decompress(DecompressorStream.java:80)
>>>        at org.apache.hadoop.io.compress.DecompressorStream.read(DecompressorStream.java:74)
>>>        at java.io.InputStream.read(InputStream.java:85)
>>>        at org.apache.hadoop.util.LineReader.readLine(LineReader.java:134)
>>>        at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.nextKeyValue(LineRecordReader.java:97)
>>>        at org.apache.pig.builtin.PigStorage.getNext(PigStorage.java:109)
>>>        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigRecordReader.nextKeyValue(PigRecordReader.java:187)
>>>        at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:423)
>>>        at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)
>>>        at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)
>>>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:621)
>>>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>>>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
>>>
>>>
>>> Thanks,
>>> Prashant
>>
>>
>>
>> --
>> Harsh J

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB