Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro + Snappy changing blocksize of snappy compression


Copy link to this message
-
Re: Avro + Snappy changing blocksize of snappy compression
Hey Nikhil,

When using Avro Datafiles, you perhaps need to tweak its sync-interval
to affect compression chunk sizes:
http://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/file/DataFileWriter.html#setSyncInterval(int)

On Wed, Apr 18, 2012 at 10:53 PM, snikhil0 <[EMAIL PROTECTED]> wrote:
> I am experimenting with Avro and snappy and want to plot the size of the
> compressed avro datafile as a function of varying compression block size. I
> am doing this by setting the configuration value for
> "io.compression.codec.snappy.buffersize". Unfortunately, this is not
> working: or more precisely for buffer sizes 256K to 2MB I get the same size
> output avro (snappyfied) data file. What am I missing? Someone had success
> with this?
>
> Thanks,
> Nikhil
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Avro-Snappy-changing-blocksize-of-snappy-compression-tp3920732p3920732.html
> Sent from the Avro - Users mailing list archive at Nabble.com.

--
Harsh J