Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Re: task jvm bootstrapping via distributed cache


Copy link to this message
-
Re: task jvm bootstrapping via distributed cache
Stan Rosenberg 2012-08-01, 15:12
On Tue, Jul 31, 2012 at 7:26 PM, Michael Segel
<[EMAIL PROTECTED]> wrote:
> Hi Stan,
>
> If I understood your question... you want to ship a jar to the nodes where the task will run prior to the start of the task?
>
> Not sure what it is you're trying to do...
> Your example isn't  really clear.

Correct.  I want to ship a jar to the task, but I need to know its
absolute path before the task jvm is launched.
As an example, -javaagent JVM option expects a jar path.

>
> See: http://hadoop.apache.org/common/docs/r0.20.2/api/org/apache/hadoop/filecache/DistributedCache.html
>
> When you pull stuff out of the cache you get the path to the jar.
> Or you should be able to get it.
>

It would be too late at that point; the task tracker controls the
launching of the JVM.  The path of the shipped jar need to be
available before the task is launched.

> Can you give a better example, there may be a different way to handle this...
>
Does the example above make sense?