Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Warnings?


Copy link to this message
-
Re: Warnings?
Hi Kevin,

Please see my comments inline,
On Sat, Apr 27, 2013 at 11:24 AM, Kevin Burton <[EMAIL PROTECTED]>wrote:

> Is the native library not available for Ubuntu? If so how do I load it?
>
Native libraries usually requires recompile, for more information please
refer Native Libraries<http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/NativeLibraries.html>
.
> Can I tell which key is off? Since I am just starting I would want to be
> as up to date as possible. It is out of date probably because I copied my
> examples from books and tutorials.
>
> I think the warning messages are telling it already, "xxx is deprecated,
use xxx instead...". In fact, most of the configure keys are changed from
hadoop 1.x to 2.x. The compatibility change may later documented on
http://wiki.apache.org/hadoop/Compatibility.
> The main class does derive from Tool. Should I ignore this warning as it
> seems to be in error?
>
Of course you can ignore this warning as long as you don't use hadoop
generic options.
>
> Thank you.
>
> On Apr 26, 2013, at 7:49 PM, Ted Xu <[EMAIL PROTECTED]> wrote:
>
> Hi,
>
> First warning is saying hadoop cannot load native library, usually a
> compression codec. In that case, hadoop will use java implementation
> instead, which is slower.
>
> Second is caused by hadoop 1.x/2.x configuration key change. You're using
> a 1.x style key under 2.x, yet hadoop still guarantees backward
> compatibility.
>
> Third is saying that the main class of a hadoop application is recommanded
> to implement org.apache.hadoop.util.Tool<http://hadoop.apache.org/docs/current/api/org/apache/hadoop/util/Tool.html>,
> or else generic command line options (e.g., -D options) will not supported.
>
>
>
> On Sat, Apr 27, 2013 at 5:51 AM, <[EMAIL PROTECTED]> wrote:
>
>> I am running a simple WordCount m/r job and I get output but I get five
>> warnings that I am not sure if I should pay attention to:
>>
>> *13/04/26 16:24:50 WARN util.NativeCodeLoader: Unable to load
>> native-hadoop library for your platform... using builtin-java classes where
>> applicable *
>>
>> *13/04/26 16:24:50 WARN conf.Configuration: session.id is deprecated.
>> Instead, use dfs.metrics.session-id *
>>
>> *13/04/26 16:24:50 WARN mapred.JobClient: Use GenericOptionsParser for
>> parsing the arguments. Applications should implement Tool for the same. *
>>
>> *13/04/26 16:24:51 WARN mapreduce.Counters: Group
>> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
>> org.apache.hadoop.mapreduce.TaskCounter instead *
>>
>> *13/04/26 16:24:51 WARN mapreduce.Counters: Counter name MAP_INPUT_BYTES
>> is deprecated. Use FileInputFormatCounters as group name and  BYTES_READ as
>> counter name instead*
>>
>> Any ideas on what these mean? The only one that I can see in the code is
>> the third one. I am using GenericOptionsParser as it is part of an example
>> that I copied. But I don't know why this is considered bad.
>>
>> Thank you.
>>
>>
>
>
> --
> Regards,
> Ted Xu
>
>
--
Regards,
Ted Xu