Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Re: Reading and Writing Sequencefile using Hadoop 2.0 Apis


Copy link to this message
-
Re: Reading and Writing Sequencefile using Hadoop 2.0 Apis
you can use if even if it's depracated.

I can find in
the org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.java,

  @Override
  public void initialize(InputSplit split,
                         TaskAttemptContext context
                         ) throws IOException, InterruptedException {
    FileSplit fileSplit = (FileSplit) split;
    conf = context.getConfiguration();
    Path path = fileSplit.getPath();
    FileSystem fs = path.getFileSystem(conf);
    this.in = new SequenceFile.Reader(fs, path, conf);
    this.end = fileSplit.getStart() + fileSplit.getLength();

    if (fileSplit.getStart() > in.getPosition()) {
      in.sync(fileSplit.getStart());                  // sync to start
    }

    this.start = in.getPosition();
    more = start < end;
  }
On Thu, Apr 18, 2013 at 6:44 AM, sumit ghosh <[EMAIL PROTECTED]> wrote:

>    **
>   I am looking for an example which is using the new Hadoop 2.0 API to
> read and write Sequence Files. Effectively I need to know how to use these
> functions:
>
> createWriter(Configuration conf, org.apache.hadoop.io.SequenceFile.Writer.Option... opts)
>
> The Old definition is not working for me:
>
> SequenceFile.createWriter( fs, conf, path, key.getClass(), value.getClass());
>
>
> Similarly I need to know what will be the code for reading the Sequence
> file, as the follwoing is deprecated:
>
> SequenceFile.Reader(fs, path, conf);
>
> Thanks,
> Sumit
>