Hello,

Let my Key-Value be something like BinaryWritables (my own class, but
something like this).  Is there a way to create the Sequence File
composed of several such key - values, without using Java?

Background:

I create objects using protocol buffers, my key and values are
serialized versions of these protocol buffer messages. These hadoop k-v
pairs that are exchanged in the mapreduce (and stored in both output and
input) are the serialized versions of these.

I would like to directly create sequence files using C++
and was curious if there is way to do this outside Java (and not have to
use JNI), as currently, its best to use a mapreduce job to convert my
textfiles to sequence files.

Thank you
Saptarshi

On Fri, Nov 27, 2009 at 7:07 PM, Saptarshi Guha <[EMAIL PROTECTED]>wrote:

Let my Key-Value be something like BinaryWritables (my own class, but
> something like this).  Is there a way to create the Sequence File
> composed of several such key - values, without using Java?
>

There is not a C++ implementation of SequenceFiles. (If you write one,
please consider contributing it back.)

A different approach would make a map only Pipes (C++) MapReduce program
that reads the data and uses SequenceFileOutputFormat for its output. The
map can emit key/value pairs as std::strings containing the bytes you want
to write.

-- Owen