Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Can I perfrom a MR on my local filesystem


Copy link to this message
-
Re: Can I perfrom a MR on my local filesystem
Nikhil, may be this blog will help you understand it how s3 is used

http://www.technology-mania.com/2012/05/s3-instead-of-hdfs-with-hadoop_05.html
http://www.technology-mania.com/2011/05/s3-as-input-or-output-for-hadoop-mr.html

note s3 is storage house and it does not have any compute capacity to run
map reduce jobs

your map reduce jobs are always running on ec2 nodes (are you looking at
how the data is accessed by mapreduce jobs while s3 as filesystem? )
On Sun, Feb 17, 2013 at 11:52 AM, Agarwal, Nikhil <[EMAIL PROTECTED]
> wrote:

>  Hi,
>
> ****
>
> Recently I followed a blog to run Hadoop on a single node cluster.****
>
> I wanted to ask that in a single node set-up of Hadoop is it necessary to
> have the data copied into Hadoop’s HDFS before running a MR on it. Can I
> run MR on my local file system too without copying the data to HDFS? ****
>
> In the Hadoop source code I saw there are implementations of other file
> systems too like S3, KFS, FTP, etc. so how does exactly a MR happen on S3
> data store ? How does JobTracker or Tasktracker run in S3 ? ****
>
> ** **
>
> I would be very thankful to get a reply to this.****
>
> ** **
>
> Thanks & Regards,****
>
> Nikhil****
>
> ** **
>

--
Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB