-
Re: flume 1.2.0 bzip2 codec problemJagadish Bihani 2012-07-26, 06:28
Thanks Jarek.
Indeed it was missing from config file. Adding it did the trick.
On 07/25/2012 08:24 PM, Jarek Jarcec Cecho wrote:
> Hi Jagadish,
> I believe that flume is using hadoop framework to get list of compression codecs. Is bzip2 codec available in your hadoop environment?
>
> You might want to check configuration property io.compression.codecs for inclusion of org.apache.hadoop.io.compress.BZip2Codec.
>
> Jarcec
>
> On Jul 25, 2012, at 12:20 PM, Jagadish Bihani wrote:
>
>> Hi
>>
>> I have downloaded and deployed latest flume code from repository
>> "https://svn.apache.org/repos/asf/flume/trunk/"
>>
>> In the conf file I am using following properties for the hdfs sink:
>>
>>
>> agent.sinks.hdfsSink.type = hdfs
>> agent.sinks.hdfsSink.hdfs.path= <xyz>
>> agent.sinks.hdfsSink.hdfs.fileType =CompressedStream
>> agent.sinks.hdfsSink.hdfs.filePrefix=<xxx>
>> agent.sinks.hdfsSink.hdfs.rollSize=0
>> agent.sinks.hdfsSink.hdfs.codeC=bzip2
>> agent.sinks.hdfsSink.hdfs.rollCount=0
>> agent.sinks.hdfsSink.hdfs.batchSize=1
>> agent.sinks.hdfsSink.hdfs.rollInterval=1
>>
>> After running it gives me an exception for bzip.
>>
>> java.lang.IllegalArgumentException: Unsupported compression codec bzip2. Please choose from: [None, GzipCodec, DefaultCodec, LzoCodec, LzopCodec, SnappyCodec]
>>
>> Isn't flume 1.2.0 supporting bzip2?
>>
>> Regards,
>> Jagadish
