Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: task jvm bootstrapping via distributed cache


Copy link to this message
-
Re: task jvm bootstrapping via distributed cache
On Tue, Jul 31, 2012 at 7:26 PM, Michael Segel
<[EMAIL PROTECTED]> wrote:
> Hi Stan,
>
> If I understood your question... you want to ship a jar to the nodes where the task will run prior to the start of the task?
>
> Not sure what it is you're trying to do...
> Your example isn't  really clear.

Correct.  I want to ship a jar to the task, but I need to know its
absolute path before the task jvm is launched.
As an example, -javaagent JVM option expects a jar path.

>
> See: http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/filecache/DistributedCache.html
>
> When you pull stuff out of the cache you get the path to the jar.
> Or you should be able to get it.
>

It would be too late at that point; the task tracker controls the
launching of the JVM.  The path of the shipped jar need to be
available before the task is launched.

> Can you give a better example, there may be a different way to handle this...
>
Does the example above make sense?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB