Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Doubt regarding use of HTTP during Shuffle phase


+
Pavan Kulkarni 2012-06-27, 16:30
+
Pavan Kulkarni 2012-06-27, 16:33
+
Owen OMalley 2012-06-27, 16:40
+
Pavan Kulkarni 2012-06-27, 16:44
Copy link to this message
-
Re: Doubt regarding use of HTTP during Shuffle phase
On Wed, Jun 27, 2012 at 9:44 AM, Pavan Kulkarni <[EMAIL PROTECTED]> wrote:
> Yes you are correct, but we can use Lustre FS and it does scale right?
> I am new to this so please excuse if I am wrong in some assumptions.

You can use hybrid approaches, but you'd be venturing into the unknown.
Take a look at these projects first:
    http://code.google.com/p/sailfish/
    http://www.xyratex.com/pdfs/whitepapers/Xyratex_white_paper_MapReduce_1-4.pdf

Thanks,
Roman.
+
Pavan Kulkarni 2012-06-27, 16:57
+
Owen OMalley 2012-06-27, 17:21
+
Pavan Kulkarni 2012-06-27, 17:30