Option 2 is better.
Please see this in SequenceFile:
public static Writer
createWriter(FileSystem fs, Configuration conf, Path name,
Class keyClass, Class valClass, int bufferSize,
short replication, long blockSize,
CompressionType compressionType, CompressionCodec codec,
Progressable progress, Metadata metadata) throws
On Thu, Feb 17, 2011 at 1:16 PM, Mapred Learn <[EMAIL PROTECTED]>wrote:
> I have a use case to upload some tera-bytes of text files as sequences
> files on HDFS.
> These text files have several layouts ranging from 32 to 62 columns
> What would be a good way to upload these files along with their metadata:
> i) creating a key, value class per text file layout and use it to create
> and upload as sequence files ?
> ii) create SequenceFile.Metadata header in each file being uploaded as
> sequence file individually ?
> Any inputs are appreciated !