Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Setting up a Hadoop client in OSGI bundle


Copy link to this message
-
Setting up a Hadoop client in OSGI bundle
Hi all,

I'm trying to setup an Hadoop client for job submissions (and more) as an
OSGI bundle.

I came over a lot of hardships but I'm kinda stuck now.

When I create a new Job for submission I setClassLoader() for the Job
Configuration  so that it would use the bundle's ClassLoader (felix), but
when I submit the job and it tries to load the splits it looks for
DefaultCodec and fails to find that because somewhere along the way,
JobClient creates a new JobContext(Configuration) but although using the
Configuration containing the bundle's ClassLoader, the jobContext
constructor doesn't copy the ClassLoader set in Configuration.
And I guess that could happen in many other places in the Hadoop frameworl.

does anyone have any experience with submitting mapreduce jobs from OSGI
bundle client ? Any tips for this problems (and others I've yet to
encounter) ?

BTW, running Hadoop 1.0.4 and HBase 0.94.2.

Thanks,

Amit.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB