Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Is it necessary to cache metadata in client side?


Copy link to this message
-
Is it necessary to cache metadata in client side?
Jeff Zhang 2010-06-11, 06:43
Hi all,

According the GFS paper claims, GFS will cache meta data in client.
But when I check the source code of hadoop, it seems that hadoop won't
cache it in client side. I just wan to make sure whether I am right ?
And wondering whether there's someone work on it ? One advantage of
caching metadata in client side I can think of is that tasktracker
will fetch job.xml in HDFS. And most of time we will run multiple task
in one node, so if tasktrack cache the metadata, it can reduce the
communication with namenode.

--
Best Regards

Jeff Zhang