Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro + Snappy changing blocksize of snappy compression

Copy link to this message
Re: Avro + Snappy changing blocksize of snappy compression
Try a range from smaller block sizes (4k) and up.  256K is a larger block
size than many compression codecs are sensitive to.

Also for reference, try it with the deflate codec at a couple different
compression levels -- 1, 3, 5, and 7 should show a trend with respect to
block size.  As the compression level increases, the compressor can take
advantage of larger blocks.

In the deflate/gzip case that I have explored heavily, the effectiveness
of the block size also varies significantly depending on the
characteristics of the data being compressed.
(note: gzip uses deflate compression)

On 4/18/12 1:33 PM, "snikhil0" <[EMAIL PROTECTED]> wrote:

>I had tried the sync Interval as well and I get the same results: meaning
>change in final avro data file.
>View this message in context:
>Sent from the Avro - Users mailing list archive at Nabble.com.