Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> read a compressed avro file


+
Weishung Chung 2012-03-31, 05:33
+
Miki Tebeka 2012-03-31, 05:44
+
Weishung Chung 2012-03-31, 06:10
Copy link to this message
-
Re: read a compressed avro file
Reading an Avro file is the same for all compression types. The header of
the file contains what compression codec is used and the schema for the data
within it.

A basic read may look like:

DataFileReader<Object> reader = new DataFileReader<Object>(myFile, new
GenericDatumReader<Object>());
try {
  for(Object datum : reader) {
    // do something with the datum
  }
} finally {
  reader.close();
}
The documentation is here:
http://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/file/DataFileRead
er.html
or
http://avro.apache.org/docs/1.6.3/api/java/org/apache/avro/file/DataFileStre
am.html

Several examples can be found in the unit tests named TestDataFile*.java:
http://svn.apache.org/viewvc/avro/tags/release-1.6.3/lang/java/avro/src/test
/java/org/apache/avro/
On 3/30/12 11:10 PM, "Weishung Chung" <[EMAIL PROTECTED]> wrote:

> java
>
> On Sat, Mar 31, 2012 at 12:44 AM, Miki Tebeka <[EMAIL PROTECTED]> wrote:
>>> > Could you guide me to an example on how to read a compressed avro file
>>> > (snappy) ?
>> In which programming language?
>
+
Weishung Chung 2012-04-03, 06:18