MapReduce, mail # user - Re: How do find the version of Hadoop inside a running Map or reduce task

How do find the version of Hadoop inside a running Map or reduce task

Steve Lewis 2012-11-05, 20:54

I need to determine what version of Hadoop is running - say under AWS - I
really want to use an API or properties in the running code but do not know
how - any ideas

--
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com

Re: How do find the version of Hadoop inside a running Map or reduce task

Lewis John Mcgibbney 2012-11-05, 21:01

I've never done this but you can try poking around with

http://hadoop.apache.org/docs/r1.0.3/api/index.html?org/apache/hadoop/HadoopVersionAnnotation.html

hth

Lewis

On Mon, Nov 5, 2012 at 8:54 PM, Steve Lewis <[EMAIL PROTECTED]> wrote:
> I need to determine what version of Hadoop is running - say under AWS - I
> really want to use an API or properties in the running code but do not know
> how - any ideas
>
> --
> Steven M. Lewis PhD
> 4221 105th Ave NE
> Kirkland, WA 98033
> 206-384-1340 (cell)
> Skype lordjoe_com
>
>

--
Lewis

Re: How do find the version of Hadoop inside a running Map or reduce task

David Rosenstrauch 2012-11-05, 21:02

On 11/05/2012 03:54 PM, Steve Lewis wrote:
> I need to determine what version of Hadoop is running - say under AWS - I
> really want to use an API or properties in the running code but do not know
> how - any ideas

Probably not the best way, but one possible way: make a call to
Runtime.exec() and call "hadoop -version".

DR

Re: How do find the version of Hadoop inside a running Map or reduce task

David Rosenstrauch 2012-11-05, 21:08

On 11/05/2012 04:02 PM, David Rosenstrauch wrote:
> On 11/05/2012 03:54 PM, Steve Lewis wrote:
>> I need to determine what version of Hadoop is running - say under AWS - I
>> really want to use an API or properties in the running code but do not
>> know
>> how - any ideas
>
> Probably not the best way, but one possible way: make a call to
> Runtime.exec() and call "hadoop -version".
>
> DR

Sorry, make that "hadoop version".

Also, FYI, it looks like that command just calls out to methods in
http://hadoop.apache.org/docs/r0.20.2/api/org/apache/hadoop/util/VersionInfo.html
. So you can probably query those directly.

DR

Re: How do find the version of Hadoop inside a running Map or reduce task

Steve Lewis 2012-11-05, 21:51

Thanks - that works perfectly

The following code reports the version as a counter under "Performance"
   // sneaky trick to extract the version
   String version = VersionInfo.getVersion();
   context.getCounter("Performance", "Version-" + version
).increment(1);

On Mon, Nov 5, 2012 at 1:08 PM, David Rosenstrauch <[EMAIL PROTECTED]>wrote:

> On 11/05/2012 04:02 PM, David Rosenstrauch wrote:
>
>> On 11/05/2012 03:54 PM, Steve Lewis wrote:
>>
>>> I need to determine what version of Hadoop is running - say under AWS - I
>>> really want to use an API or properties in the running code but do not
>>> know
>>> how - any ideas
>>>
>>
>> Probably not the best way, but one possible way: make a call to
>> Runtime.exec() and call "hadoop -version".
>>
>> DR
>>
>
> Sorry, make that "hadoop version".
>
> Also, FYI, it looks like that command just calls out to methods in
> http://hadoop.apache.org/docs/**r0.20.2/api/org/apache/hadoop/**
> util/VersionInfo.html<http://hadoop.apache.org/docs/r0.20.2/api/org/apache/hadoop/util/VersionInfo.html>. So you can probably query those directly.
>
> DR
>
>
--
Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com