Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Cuda Program in Hadoop Cluster


Copy link to this message
-
Re: Cuda Program in Hadoop Cluster
A Hadoop/CUDA example would be great, and probably of general interest
to the list.

Russ
Sent from my iPhone

On Mar 13, 2011, at 4:14 AM, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

> Hello Adarsh,
>
> At the time I was doing this integration, I used Pipes because it's C++
> based, which is easier for porting CUDA C code I assume.
>
> It actually is quite straightforward I think. What I have done can be
> summarized into:
>
> 1) a simple abstract class that implements the common protocol for
> Mapper/Reducer such that each CUDA mapper/reducer does not have to deal
> with the Pipes communication. They just launch CUDA kernel and collect
> results.
>
> 2) a set of simple shell scripts for uploading the mapper/reducer/CUDA
> kernel to the Hadoop DFS and kickstarts the runs.
>
> 3) modify the CUDA program into a mapper inheriting from the wrapper class
> so that it takes input from Pipes and dump results also to Pipes.
>
> If you want, I can try to find the code for you but I think you get the
> idea. I didn't do a whole lot because it's mainly a class project. We just
> want to demonstrate its feasibility and power saving.
>
> Thank you for your interest in our work! Let me know if you need further
> assistance.
>
>
> Peng
>> Thanks Chen for your guidance,
>>
>> To translate CUDA in Hadoop environment, following are the options :
>>
>> -- Hadoop Streaming: Standard I/O ( Recommended by you )
>> -- Hadoop Pipes: C++ library, Socket connection, ( I'm looking for )
>> -- JNI, JNI-based CUDA wrapper (JCUDA) ( Done and able to run a Jcuda
>> Program in Hadoop Cluster )
>>
>> We use Hadoop Pipes for our proposed technique
>>    . MapReduce applications/CUDA kernel ? written in C++
>>
>> So i.e why I am trying a C Cuda code to run through Hadoop Pipes, is it
>> difficult or not possible.
>> I am looking for a demo Cuda program that is up and running in Hadoop
>> Cluster that clarifies my  basic concepts so that I program accordingly
>> in future.
>>
>>
>> Looking forward for some more guidance.
>>
>> Thanks once again for your wishes .
>>
>>
>> With best Regards,
>>
>> Adarsh Sharma
>>
>>
>>
>> He Chen wrote:
>>> Hi, Adarsh Sharma
>>>
>>> For C code
>>>
>>> My friend employ hadoop streaming to run CUDA C code. You can send email
>>> to
>>> him. [EMAIL PROTECTED].
>>>
>>> Best wishes!
>>>
>>> Chen
>>>
>>>
>>> On Thu, Mar 3, 2011 at 11:18 PM, Adarsh Sharma
>>> <[EMAIL PROTECTED]>wrote:
>>>
>>>
>>>> Dear all,
>>>>
>>>> I followed a fantastic tutorial and able to run the Wordcont C++
>>>> Program in
>>>> Hadoop Cluster.
>>>>
>>>>
>>>> http://cs.smith.edu/dftwiki/index.php/Hadoop_Tutorial_2.2_--_Running_C%2B%2B_Programs_on_Hadoop
>>>>
>>>> But know I want to run a Cuda Program in the Hadoop Cluster but results
>>>> in
>>>> errors.
>>>> Is anyone has done it before and guide me how to do this.
>>>>
>>>> I attached the both files. Please find the attachment.
>>>>
>>>>
>>>> Thanks & best Regards,
>>>>
>>>> Adarsh Sharma
>>>>
>>>>
>>>
>>>
>>
>>
>
>