Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> configuration and FileSystem


Copy link to this message
-
Re: configuration and FileSystem

On May 10, 2011, at 9:57 AM, Gang Luo wrote:
> I was confused by the configuration and file system in hadoop. when we create a
> FileSystem object and read/write something through it, are we writing to or
> reading from HDFS?

Typically, yes.

> Could it be local file system?

Yes.
> If yes, what determines which
> file system it is? Configuration object we used to create the FileSystem object?

Yes.

> When I write mapreduce program which "extends Configured implements Tool", I can
> get the right configuration by calling getConf() and use the FileSystem object
> to communicate with HDFS. What if I want to read/write HDFS in a separate
> Utility class? Where does the configuration come from?

You need to supply it.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB