|
|
-
Re: Calling C inside MRBrock Noland 2012-12-03, 15:56
Hi,
Here is an example of how to call native methods from Java in a MR context: https://github.com/brockn/hadoop-thumbnail The most important item IMHO is that you have a clear separation of concerns. Meaning that you can test the C code without java and test the C+Java without MapReduce. Brock On Mon, Dec 3, 2012 at 7:52 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote: > Hello Simone, > > Thank you so much for the pointers. I am actually looking for some way > using which this can be achieved without Streaming or Pipes (If possible at > all). > > And I did not get why did you point out to the API page. Please let me know > if you know about something that I could relate. Apologies for my ignorance. > > Many thanks. > > Regards, > Mohammad Tariq > > > > On Mon, Dec 3, 2012 at 7:11 PM, Simone Leo <[EMAIL PROTECTED]> wrote: >> >> You can use Hadoop Pipes: >> >> http://hadoop.apache.org/docs/stable/api/index.html >> http://wiki.apache.org/hadoop/C%2B%2BWordCount >> >> Simone >> >> >> On 12/03/2012 01:05 PM, Mohammad Tariq wrote: >>> >>> Hello list, >>> >>> I have a tool (written in C) that performs some different >>> types of operations and can be used as a command line utility. I had to >>> write a similar tool, as we have moved towards Hadoop platform for most >>> of the things. >>> >>> Till now I have taken this tool as reference and written MR jobs >>> corresponding to some the modules of this tool and they are working >>> fine. But I am wasting a lot of time in this. So, I just wanted to ask >>> if it is possible to call this tool through a MR job?? Somewhat like JNI >>> kinda thing. (I hope it is, otherwise I have to write rest of things >>> from scratch and we are running out of time). >>> >>> Many thanks. >>> >>> Regards, >>> Mohammad Tariq >>> >> >> -- >> Simone Leo >> Data Fusion - Distributed Computing >> CRS4 >> POLARIS - Building #1 >> Piscina Manna >> I-09010 Pula (CA) - Italy >> e-mail: [EMAIL PROTECTED] >> http://www.crs4.it > > -- Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/ |