Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> building a department GPU cluster


+
Roberto Nunnari 2013-01-17, 08:24
Copy link to this message
-
Re: building a department GPU cluster
Hadoop streaming can do this, and there's been some discussion in the past,
but it's not a core use case. Check the list archives.

Russell Jurney http://datasyndrome.com

On Jan 17, 2013, at 9:25 AM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:

I don't think running hadoop on a GPU cluster is a common use case; the
types of workloads for a hadoop vs. gpu cluster are very different although
a quick google search did turn up some. So this is probably not the best
mailing list for your question.

J
On Thu, Jan 17, 2013 at 5:18 AM, Roberto Nunnari
<[EMAIL PROTECTED]>wrote:

> Roberto Nunnari wrote:
>
>> Hi all.
>>
>> I'm writing to you to ask for advice or a hint to the right direction.
>>
>> In our department, more and more researchers ask us (IT administrators)
>> to assemble (or to buy) GPGPU powered workstations to do parallel computing.
>>
>> As I already manage a small CPU cluster (resources managed using SGE),
>> with my boss we talked about building a new GPU cluster. The problem is
>> that I have no experience at all with GPU clusters.
>>
>> Apart from the already running GPU workstations, we already have some new
>> HW that looks promising to me as a starting point for a GPU cluster.
>>
>> - 1x Dell PowerEdge R720
>> - 1x Dell PowerEdge C410x
>> - 1x NVIDIA M2090 PCIe x16
>> - 1x NVIDIA iPASS Cable Kit
>> (Dell forgot to include the iPASS adapter for the R720!! :-D)
>>
>> I'd be grateful if you could kindly give me some advice and/or hint to
>> the right direction.
>>
>> In particular I'm interested on your opinion on:
>> 1) is the above HW suitable for a small (2 to 4/6 GPUs) GPU cluster?
>> 2) is apache adhoop suitable (or what could we use?) as a queuing and
>> resource management system? We would like the cluster to be usable by many
>> users at once in a way that no user has to worry about resources, just like
>> we do on the CPU cluster with SGE.
>> 3) What distribution of linux would be more appropriate?
>> 4) necessary stack of sw? (cuda, hadoop, other?)
>>
>> Thank you very much for your valuable insight!
>>
>> Best regards.
>> Robi
>>
>
> Anybody on this, please?
> Robi
>
>
+
Thiago Vieira 2013-01-18, 03:37