-Re: Mapreduce outputs directly to SortedKeyValueFile
Jeremy Lewi 2012-07-21, 19:44
I filed https://issues.apache.org/jira/browse/AVRO-1130.
Unfortunately I don't think I will be able to work on this.
On Sat, Jul 21, 2012 at 12:30 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> Hi Jeremy,
> There's no OutputFormat for directly writing the SortedKeyValueFile in
> the avro-mapred package yet, but it can certainly be written by
> you/added to avro. I don't see why you can't write the sorted file
> right from your job (from a reducer, I'm assuming, for it is sorted).
> You merely need to extend the OutputFormat and use a
> SortedKeyValueFile writer instead of a simple DataFile writer (as
> AvroOutputFormat's getRecordWriter provides/does). Please do file an
> AVRO JIRA for this, as its a hole in what Avro provides that needs to
> be filled.
> On Sun, Jul 22, 2012 at 12:50 AM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
> > Hi avro-users,
> > Is it possible for my mapreduce job to write directly to a SortedKeyValue
> > file? Or must I first output to a regular avro file and then build the
> > index?
> > Thanks
> > J
> Harsh J