Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Reading and Writing Sequencefile using Hadoop 2.0 Apis


Copy link to this message
-
Re: Reading and Writing Sequencefile using Hadoop 2.0 Apis
you can use if even if it's depracated.

I can find in
the org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.java,

  @Override
  public void initialize(InputSplit split,
                         TaskAttemptContext context
                         ) throws IOException, InterruptedException {
    FileSplit fileSplit = (FileSplit) split;
    conf = context.getConfiguration();
    Path path = fileSplit.getPath();
    FileSystem fs = path.getFileSystem(conf);
    this.in = new SequenceFile.Reader(fs, path, conf);
    this.end = fileSplit.getStart() + fileSplit.getLength();

    if (fileSplit.getStart() > in.getPosition()) {
      in.sync(fileSplit.getStart());                  // sync to start
    }

    this.start = in.getPosition();
    more = start < end;
  }
On Thu, Apr 18, 2013 at 6:44 AM, sumit ghosh <[EMAIL PROTECTED]> wrote:

>    **
>   I am looking for an example which is using the new Hadoop 2.0 API to
> read and write Sequence Files. Effectively I need to know how to use these
> functions:
>
> createWriter(Configuration conf, org.apache.hadoop.io.SequenceFile.Writer.Option... opts)
>
> The Old definition is not working for me:
>
> SequenceFile.createWriter( fs, conf, path, key.getClass(), value.getClass());
>
>
> Similarly I need to know what will be the code for reading the Sequence
> file, as the follwoing is deprecated:
>
> SequenceFile.Reader(fs, path, conf);
>
> Thanks,
> Sumit
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB