Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Reading from sequence file using java FS api


+
Mohit Anchlia 2012-11-12, 23:16
+
Mohit Anchlia 2012-11-13, 00:37
Copy link to this message
-
Re: Reading from sequence file using java FS api
Yes, the codec information is stored in the file's header.

Same goes for Avro where even the deserialization schema logic is
stored in addition, so you can just directly read into usable
primitive/compound objects and not have to do manual transformation
work.

On Tue, Nov 13, 2012 at 6:07 AM, Mohit Anchlia <[EMAIL PROTECTED]> wrote:
> I was simple able to read using below code. Didn't have to decompress. It
> looks like reader automatically knows and decompresses the file before
> returning it to the user.
>
>
> On Mon, Nov 12, 2012 at 3:16 PM, Mohit Anchlia <[EMAIL PROTECTED]>
> wrote:
>>
>> I am looking for an example that read snappy compressed snappy file. Could
>> someone point me to it? What I have so far is this:
>>
>>
>> Configuration conf >>
>> new Configuration();
>>
>> FileSystem fs = FileSystem.get(URI.create(uri), conf);
>>
>> Path path >>
>> new Path(uri);
>>
>> SequenceFile.Reader reader >>
>> null;
>>
>> org.apache.hadoop.io.LongWritable key >>
>> new org.apache.hadoop.io.LongWritable();
>>
>> org.apache.hadoop.io.Text value >>
>> new org.apache.hadoop.io.Text();
>>
>> try {
>>
>> reader = new SequenceFile.Reader(fs, path, conf);
>
>

--
Harsh J