Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Hadoop MapReduce


Copy link to this message
-
Re: Hadoop MapReduce
MR has a "local mode" that does what you want.  Pig has the ability to use this mode.  I did a quick search but didn't immediately find a good link to documentation, but hopefully this gets you going in the right direction.

Daryn

On Apr 22, 2013, at 6:01 PM, David Gkogkritsiani wrote:

Helllo,

I have undertaken my diploma thesis on Hadoop MapReduce and I have been requested to I do an application written in MapReduce.
I found on internet this code and I ran the code :

http://paste.ubuntu.com/5591999/

How can I add the code to stores the pages somewhere locally (text only, not Images) and then have to be processed . ie,I should a Mapreduce code, which would download pages from the web and store on the local file system and not the HDFS.
After ,I run the quest (program) in order to not depend on network speed.

Because ,my network is so slow.

I do this to improvement performance.

I am running Hadoop Version 0.20.2 .
I am new to Hadoop and am kinda lost and any help would be greatly appreciated.

Thanks in advance for any assistance !