Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> getting null from CompressionCodecFactory.getCodec(Path file)


Copy link to this message
-
Re: getting null from CompressionCodecFactory.getCodec(Path file)
> I got it. For some reason getDefaultExtension() returns  
> ".lzo_deflate".
>
> Is that a bug? Shouldn't it be .lzo?

The .lzo suffix is reserved for lzop (LzopCodec). LzoCodec doesn't  
generate compatible output, hence "lzo_deflate". -C
>
> In the head revision I couldn't find it at all in
> http://svn.apache.org/repos/asf/hadoop/core/trunk/src/core/org/apache/hadoop/io/compress/
>
> There should be a Class LzoCodec.java. Was that moved to somewhere  
> else?
>
> Gert
>
> Gert Pfeifer wrote:
>> Arun C Murthy wrote:
>>> On Jan 13, 2009, at 7:29 AM, Gert Pfeifer wrote:
>>>
>>>> Hi,
>>>> I want to use an lzo file as input for a mapper. The record reader
>>>> determines the codec using a CompressionCodecFactory, like this:
>>>>
>>>> (Hadoop version 0.19.0)
>>>>
>>> http://hadoop.apache.org/core/docs/r0.19.0/native_libraries.html
>>
>> I should have mentioned that I have these native libs running:
>> 2009-01-14 10:00:21,107 INFO org.apache.hadoop.util.NativeCodeLoader:
>> Loaded the native-hadoop library
>> 2009-01-14 10:00:21,111 INFO org.apache.hadoop.io.compress.LzoCodec:
>> Successfully loaded & initialized native-lzo library
>>
>> Is that what you tried to point out with this link?
>>
>> Gert
>>
>>> hth,
>>> Arun
>>>
>>>> compressionCodecs = new CompressionCodecFactory(job);
>>>> System.out.println("Using codecFactory:  
>>>> "+compressionCodecs.toString());
>>>> final CompressionCodec codec = compressionCodecs.getCodec(file);
>>>> System.out.println("Using codec: "+codec+" for file  
>>>> "+file.getName());
>>>>
>>>>
>>>> The output that I get is:
>>>>
>>>> Using codecFactory: { etalfed_ozl.:
>>>> org.apache.hadoop.io.compress.LzoCodec }
>>>> Using codec: null for file test.lzo
>>>>
>>>> Of course, the mapper does not work without codec. What could be  
>>>> the
>>>> problem?
>>>>
>>>> Thanks,
>>>> Gert
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB