Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Re: SDP support for Hadoop RPC


+
Milind Bhandarkar 2013-10-09, 00:57
+
Yasin Celik 2013-10-09, 01:13
Copy link to this message
-
Re: SDP support for Hadoop RPC
Have you checked out Mellanox's UDA ?

There have been a lot of efforts from RoCE providers to get Hadoop
optimized implementations with modern networking hardware. Even RedHat has
announced HPN (High Performance Networking) interfaces to allow modern
networks. It might take a few years to get there for Open Source Hadoop
implementations, but believe me, every Hadoop vendor based off of open
source Hadoop is incorporating new network architectures in their
proprietary distro. We were there first. That's all.

- milind
---
Milind Bhandarkar
Chief Scientist
Pivotal
+1-650-523-3858 (W)
+1-408-666-8483 (C)
On Tue, Oct 8, 2013 at 6:13 PM, Yasin Celik <[EMAIL PROTECTED]> wrote:

> Hello
>
> How do you get hadoop worked with infiniband? Do you use SDP?
> I only need to use RPC of hadoop for now. Any help on how to get it worked?
>
> Thanks
> Yasin
>
>
> On Tue, Oct 8, 2013 at 8:57 PM, Milind Bhandarkar <
> [EMAIL PROTECTED]
> > wrote:
>
> > Yes, we have. It works very well, but it is considered too niche by folks
> > who insist on buying the least capable hardware for their test clusters,
> > and therefore, recommend such underpowered clusters to customers as well.
> >
> > - milind
> >
> > > On Oct 8, 2013, at 16:49, Yasin Celik <[EMAIL PROTECTED]> wrote:
> > >
> > > Hello,
> > >
> > > I looked around but I think it seems there is no Sockets Direct
> > > Protocol(SDP) support in Hadoop RPC.
> > > I basically want to take advantage of Infiniband when transferring
> data.
> > > Is there any known support for this? Has anybody tried this before?
> > >
> > > Best
> > >
> > > ---------
> > > Yasin Celik
> >
>
>
>
> --
> ---------
> Yasin Celik
>
+
Steve Loughran 2013-10-09, 08:45
+
Milind Bhandarkar 2013-10-09, 09:09
+
Suresh Srinivas 2013-10-09, 15:47
+
Milind Bhandarkar 2013-10-09, 17:38
+
Steve Loughran 2013-10-09, 08:50
+
Luke Lu 2013-10-10, 08:00
+
Steve Loughran 2013-10-10, 08:41
+
Yasin Celik 2013-10-10, 12:12
+
Luke Lu 2013-10-10, 19:12