Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - How to read LZO compressed files?

Copy link to this message
How to read LZO compressed files?
edward choi 2012-01-02, 05:34

I'm having trouble trying to handle lzo compressed files.
The input files are compressed by LzopCodec provided by hadoop-lzo package.
And I am using Cloudera 3 update 2 version Hadoop.

I don't need to split the input file, so there is no need telling me to
index the input file and to use LzoTextInputFormat, unless that is the only
way to handle lzo-compressed files.

I thought all I needed to do was set the job input format as
"TextInputFormat" and hadoop will take care of the rest.
When I do that, I don't get any error messages but log files tell me that
input files are not decompressed at all. Input files are being handled as
raw text files.

Is there a specific way to read files with lzo extension?