Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Cuda Program in Hadoop Cluster


Copy link to this message
-
Re: Cuda Program in Hadoop Cluster
A Hadoop/CUDA example would be great, and probably of general interest
to the list.

Russ
Sent from my iPhone

On Mar 13, 2011, at 4:14 AM, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> Hello Adarsh,
>
> At the time I was doing this integration, I used Pipes because it's C++
> based, which is easier for porting CUDA C code I assume.
>
> It actually is quite straightforward I think. What I have done can be
> summarized into:
>
> 1) a simple abstract class that implements the common protocol for
> Mapper/Reducer such that each CUDA mapper/reducer does not have to deal
> with the Pipes communication. They just launch CUDA kernel and collect
> results.
>
> 2) a set of simple shell scripts for uploading the mapper/reducer/CUDA
> kernel to the Hadoop DFS and kickstarts the runs.
>
> 3) modify the CUDA program into a mapper inheriting from the wrapper class
> so that it takes input from Pipes and dump results also to Pipes.
>
> If you want, I can try to find the code for you but I think you get the
> idea. I didn't do a whole lot because it's mainly a class project. We just
> want to demonstrate its feasibility and power saving.
>
> Thank you for your interest in our work! Let me know if you need further
> assistance.
>
>
> Peng
>> Thanks Chen for your guidance,
>>
>> To translate CUDA in Hadoop environment, following are the options :
>>
>> -- Hadoop Streaming: Standard I/O ( Recommended by you )
>> -- Hadoop Pipes: C++ library, Socket connection, ( I'm looking for )
>> -- JNI, JNI-based CUDA wrapper (JCUDA) ( Done and able to run a Jcuda
>> Program in Hadoop Cluster )
>>
>> We use Hadoop Pipes for our proposed technique
>>    . MapReduce applications/CUDA kernel ? written in C++
>>
>> So i.e why I am trying a C Cuda code to run through Hadoop Pipes, is it
>> difficult or not possible.
>> I am looking for a demo Cuda program that is up and running in Hadoop
>> Cluster that clarifies my  basic concepts so that I program accordingly
>> in future.
>>
>>
>> Looking forward for some more guidance.
>>
>> Thanks once again for your wishes .
>>
>>
>> With best Regards,
>>
>> Adarsh Sharma
>>
>>
>>
>> He Chen wrote:
>>> Hi, Adarsh Sharma
>>>
>>> For C code
>>>
>>> My friend employ hadoop streaming to run CUDA C code. You can send email
>>> to
>>> him. [EMAIL PROTECTED].
>>>
>>> Best wishes!
>>>
>>> Chen
>>>
>>>
>>> On Thu, Mar 3, 2011 at 11:18 PM, Adarsh Sharma
>>> <[EMAIL PROTECTED]>wrote:
>>>
>>>
>>>> Dear all,
>>>>
>>>> I followed a fantastic tutorial and able to run the Wordcont C++
>>>> Program in
>>>> Hadoop Cluster.
>>>>
>>>>
>>>> http://cs.smith.edu/dftwiki/index.php/Hadoop_Tutorial_2.2_--_Running_C%2B%2B_Programs_on_Hadoop
>>>>
>>>> But know I want to run a Cuda Program in the Hadoop Cluster but results
>>>> in
>>>> errors.
>>>> Is anyone has done it before and guide me how to do this.
>>>>
>>>> I attached the both files. Please find the attachment.
>>>>
>>>>
>>>> Thanks & best Regards,
>>>>
>>>> Adarsh Sharma
>>>>
>>>>
>>>
>>>
>>
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB