Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> TFile Named Meta Blocks Read Write Related


+
Dare 2012-07-03, 01:07
Copy link to this message
-
Re: TFile Named Meta Blocks Read Write Related
For both 1 and 2, if your metadata is a Writable, you can simply reuse
its readFields() and write() methods to serialize it into and out of
the data output/input streams.

For instance, assume dos is out, and dis is in, and obj1 (K) and obj2
(V) are my writables, then I do:

To write K and V:
obj1.write(dos);
obj2.write(dos);

To read K and V in proper order (order is important when
deserializing), reconstruct your writable objects and read them in:
obj1.readFields(dis);
obj2.readFields(dis);

Does this not work for you?

On Tue, Jul 3, 2012 at 6:37 AM, Dare <[EMAIL PROTECTED]> wrote:
> Hi Hadoop Team,
>
> I have been working with the TFiles in Hadoop. I got a few questions
> regarding Named Meta Blocks.
>
> 1) Using TFile.Writer one can append a <K,V> pair. But, if I prepare a Meta
> Block, then it returns a DataOutputStream which allows to write in byte[],
> since my <K,V> pairs are serialized objects.
>     Is this the same way or is there something I am missing. Because, out of
> my understanding, if I write it as a <K,V> pair, the key indexes will be
> prepared at the tail of the TFile.
>     But, when I write it as a just byte[], i am not sure if the indexes are
> formed.
>
> 2) While reading, is there a way to read <K,V> entry using the
> DataInputStream got from the getMetaBlock().
>
> Thanks
> DaRe

--
Harsh J
+
Dare 2012-07-03, 16:56
+
Dare 2012-07-03, 19:21
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB