Jagadish Bihani 2012-07-25, 10:20
Jarek Jarcec Cecho 2012-07-25, 14:54
-Re: flume 1.2.0 bzip2 codec problem
Jagadish Bihani 2012-07-26, 06:28
Indeed it was missing from config file. Adding it did the trick.
On 07/25/2012 08:24 PM, Jarek Jarcec Cecho wrote:
> Hi Jagadish,
> I believe that flume is using hadoop framework to get list of compression codecs. Is bzip2 codec available in your hadoop environment?
> You might want to check configuration property io.compression.codecs for inclusion of org.apache.hadoop.io.compress.BZip2Codec.
> On Jul 25, 2012, at 12:20 PM, Jagadish Bihani wrote:
>> I have downloaded and deployed latest flume code from repository
>> In the conf file I am using following properties for the hdfs sink:
>> agent.sinks.hdfsSink.type = hdfs
>> agent.sinks.hdfsSink.hdfs.path= <xyz>
>> agent.sinks.hdfsSink.hdfs.fileType =CompressedStream
>> After running it gives me an exception for bzip.
>> java.lang.IllegalArgumentException: Unsupported compression codec bzip2. Please choose from: [None, GzipCodec, DefaultCodec, LzoCodec, LzopCodec, SnappyCodec]
>> Isn't flume 1.2.0 supporting bzip2?