Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - building a department GPU cluster


+
Roberto Nunnari 2013-01-17, 08:24
+
Russell Jurney 2013-01-18, 00:24
Copy link to this message
-
Re: building a department GPU cluster
Thiago Vieira 2013-01-18, 03:37
I've seen some academic researches on this direction, with good results.
Some computations can be expressed by GPGPU, but it is still a restrict
number of cases. If is not easy to solve problems using MapReduce, solve
some problems with SIMD is harder.

--
Thiago Vieira
On Thu, Jan 17, 2013 at 9:24 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:

> Hadoop streaming can do this, and there's been some discussion in the
> past, but it's not a core use case. Check the list archives.
>
> Russell Jurney http://datasyndrome.com
>
> On Jan 17, 2013, at 9:25 AM, Jeremy Lewi <[EMAIL PROTECTED]> wrote:
>
> I don't think running hadoop on a GPU cluster is a common use case; the
> types of workloads for a hadoop vs. gpu cluster are very different although
> a quick google search did turn up some. So this is probably not the best
> mailing list for your question.
>
> J
>
>
> On Thu, Jan 17, 2013 at 5:18 AM, Roberto Nunnari <[EMAIL PROTECTED]
> > wrote:
>
>> Roberto Nunnari wrote:
>>
>>> Hi all.
>>>
>>> I'm writing to you to ask for advice or a hint to the right direction.
>>>
>>> In our department, more and more researchers ask us (IT administrators)
>>> to assemble (or to buy) GPGPU powered workstations to do parallel computing.
>>>
>>> As I already manage a small CPU cluster (resources managed using SGE),
>>> with my boss we talked about building a new GPU cluster. The problem is
>>> that I have no experience at all with GPU clusters.
>>>
>>> Apart from the already running GPU workstations, we already have some
>>> new HW that looks promising to me as a starting point for a GPU cluster.
>>>
>>> - 1x Dell PowerEdge R720
>>> - 1x Dell PowerEdge C410x
>>> - 1x NVIDIA M2090 PCIe x16
>>> - 1x NVIDIA iPASS Cable Kit
>>> (Dell forgot to include the iPASS adapter for the R720!! :-D)
>>>
>>> I'd be grateful if you could kindly give me some advice and/or hint to
>>> the right direction.
>>>
>>> In particular I'm interested on your opinion on:
>>> 1) is the above HW suitable for a small (2 to 4/6 GPUs) GPU cluster?
>>> 2) is apache adhoop suitable (or what could we use?) as a queuing and
>>> resource management system? We would like the cluster to be usable by many
>>> users at once in a way that no user has to worry about resources, just like
>>> we do on the CPU cluster with SGE.
>>> 3) What distribution of linux would be more appropriate?
>>> 4) necessary stack of sw? (cuda, hadoop, other?)
>>>
>>> Thank you very much for your valuable insight!
>>>
>>> Best regards.
>>> Robi
>>>
>>
>> Anybody on this, please?
>> Robi
>>
>>
>