Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: executing files on hdfs via hadoop not possible? is JNI/JNA a reasonable solution?


Copy link to this message
-
Re: executing files on hdfs via hadoop not possible? is JNI/JNA a reasonable solution?
Hi Julian,

Replying to your original question.
2013/3/17 Julian Bui <[EMAIL PROTECTED]>

> Ah, thanks for clarifying, Harsh,
>
> >> On Sun, Mar 17, 2013 at 3:09 PM, Julian Bui <[EMAIL PROTECTED]>
> wrote:
>
>> >> > Hi hadoop users,
>> >> >
>> >> > I just want to verify that there is no way to put a binary on HDFS
>> and
>> >> > execute it using the hadoop java api.  If not, I would appreciate
>> advice
>> >> > in
>> >> > getting in creating an implementation that uses native libraries.
>>
>

There's support in the API for distributing native libraries or executables
and creating a symlink in the working directory of your MR processes.

This is called the distributed cache and an example can be found here:
http://wiki.apache.org/hadoop/HowToDebugMapReducePrograms#How_to_submit_debug_script

Best regards,

Jens
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB