-Re: Calling C inside MR
Brock Noland 2012-12-03, 15:56
Here is an example of how to call native methods from Java in a MR context:
The most important item IMHO is that you have a clear separation of
concerns. Meaning that you can test the C code without java and test
the C+Java without MapReduce.
On Mon, Dec 3, 2012 at 7:52 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:
> Hello Simone,
> Thank you so much for the pointers. I am actually looking for some way
> using which this can be achieved without Streaming or Pipes (If possible at
> And I did not get why did you point out to the API page. Please let me know
> if you know about something that I could relate. Apologies for my ignorance.
> Many thanks.
> Mohammad Tariq
> On Mon, Dec 3, 2012 at 7:11 PM, Simone Leo <[EMAIL PROTECTED]> wrote:
>> You can use Hadoop Pipes:
>> On 12/03/2012 01:05 PM, Mohammad Tariq wrote:
>>> Hello list,
>>> I have a tool (written in C) that performs some different
>>> types of operations and can be used as a command line utility. I had to
>>> write a similar tool, as we have moved towards Hadoop platform for most
>>> of the things.
>>> Till now I have taken this tool as reference and written MR jobs
>>> corresponding to some the modules of this tool and they are working
>>> fine. But I am wasting a lot of time in this. So, I just wanted to ask
>>> if it is possible to call this tool through a MR job?? Somewhat like JNI
>>> kinda thing. (I hope it is, otherwise I have to write rest of things
>>> from scratch and we are running out of time).
>>> Many thanks.
>>> Mohammad Tariq
>> Simone Leo
>> Data Fusion - Distributed Computing
>> POLARIS - Building #1
>> Piscina Manna
>> I-09010 Pula (CA) - Italy
>> e-mail: [EMAIL PROTECTED]
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/