Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: task jvm bootstrapping via distributed cache

Copy link to this message
Re: task jvm bootstrapping via distributed cache
On Tue, Jul 31, 2012 at 7:26 PM, Michael Segel
> Hi Stan,
> If I understood your question... you want to ship a jar to the nodes where the task will run prior to the start of the task?
> Not sure what it is you're trying to do...
> Your example isn't  really clear.

Correct.  I want to ship a jar to the task, but I need to know its
absolute path before the task jvm is launched.
As an example, -javaagent JVM option expects a jar path.

> See: http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/filecache/DistributedCache.html
> When you pull stuff out of the cache you get the path to the jar.
> Or you should be able to get it.

It would be too late at that point; the task tracker controls the
launching of the JVM.  The path of the shipped jar need to be
available before the task is launched.

> Can you give a better example, there may be a different way to handle this...
Does the example above make sense?