Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Re: Reading and Writing Sequencefile using Hadoop 2.0 Apis


Copy link to this message
-
Re: Reading and Writing Sequencefile using Hadoop 2.0 Apis
sumit ghosh 2013-04-20, 02:36
Hi,
 
Looks like it still points to the old API. The following worked for me -
http://stackoverflow.com/questions/16070587/reading-and-writing-sequencefile-using-hadoop-2-0-apis
 
    String uri = args[0];
    Configuration conf = new Configuration();
    Path path = new Path( uri);

    IntWritable key = new IntWritable();
    Text value = new Text();

    CompressionCodec Codec = new GzipCodec();
    SequenceFile.Writer writer = null;
    Option optPath = SequenceFile.Writer.file(path);
    Option optKey = SequenceFile.Writer.keyClass(key.getClass());
    Option optVal = SequenceFile.Writer.valueClass(value.getClass());
    Option optCom = SequenceFile.Writer.compression(CompressionType.RECORD,  Codec);

        writer = SequenceFile.createWriter( conf, optPath, optKey, optVal, optCom);

 
Thanks,
Sumit
________________________________
From: Harsh J <[EMAIL PROTECTED]>
To: "<[EMAIL PROTECTED]>" <[EMAIL PROTECTED]>; sumit ghosh <[EMAIL PROTECTED]>
Sent: Wednesday, 17 April 2013 9:52 PM
Subject: Re: Reading and Writing Sequencefile using Hadoop 2.0 Apis

Sumit,

I believe we've answered this one before, so you may find http://search-hadoop.com/m/xp2w02A8bqw1 helpful too.
On Thu, Apr 18, 2013 at 4:14 AM, sumit ghosh <[EMAIL PROTECTED]> wrote:
 I am looking for an example which is using the new Hadoop 2.0 API to read and write Sequence Files. Effectively I need to know how to use these functions:
>createWriter(Configuration conf, org.apache.hadoop.io.SequenceFile.Writer.Option... opts)
>The Old definition is not working for me:
>SequenceFile.createWriter( fs, conf, path, key.getClass(), value.getClass());
>
>Similarly I need to know what will be the code for reading the Sequence file, as the follwoing is deprecated:
>SequenceFile.Reader(fs, path, conf);  
>Thanks,
>Sumit-- Harsh J