Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Warnings?


Copy link to this message
-
Re: Warnings?
The env-var is auto-created by the "hadoop" script for you when you
invoke "hadoop jar". You do not necessarily have to manually set it,
nor do you have to compile the native libs if what you're using is
pre-built for your OS.

On Tue, Apr 30, 2013 at 12:52 AM,  <[EMAIL PROTECTED]> wrote:
> I don't have this environment  variable. Should I create it in .bashrc AND
> /etc/profile?
>
>
> On Mon, Apr 29, 2013 at 1:55 PM, Omkar Joshi wrote:
>
>  Hi,
>
> did you check in your ubuntu installation; "libhadoop" binary.. it is
> present in my ubuntu installation at a relative path of (I used apache
> installation)
>
> "hadoop-common-project/hadoop-common/target/native/target/usr/local/lib"
>
>
> if present add it to your LID_LIBRARY_PATH.
>
>
> if not present then you can try rebuilding your hadoop installation
>
>
> "mvn clean install -Pnative -Pdist -Dtar -DskipTests"
>
>
>
> Thanks, Omkar Joshi
> Hortonworks Inc.
>
>
> On Mon, Apr 29, 2013 at 11:19 AM, Kevin Burton < [EMAIL PROTECTED]>
> wrote:
> If it doesn't work what are my options? Is there source that I can download
> and compile?
>
> On Apr 29, 2013, at 10:31 AM, Ted Xu < [EMAIL PROTECTED]> wrote:
>
> Hi Kevin,
> Native libraries are those implemented using C/C++, which only provide code
> level portability (instead of binary level portability, as Java do). That is
> to say, the binaries provided by CDH4 distribution will in most cases be
> broken in your environment.
>
> To check if your native libraries are working or not, you can follow the
> instructions I sent previously. Quote as following.
>
> <blockquote>
> During runtime, check the hadoop log files for your MapReduce tasks.
>
>    • If everything is all right, then:  DEBUG util.NativeCodeLoader - Trying
> to load the custom-built native-hadoop library...   INFO
> util.NativeCodeLoader - Loaded the native-hadoop library
>    • If something goes wrong, then:  INFO util.NativeCodeLoader - Unable to
> load native-hadoop library for your platform... using builtin-java classes
> where applicable
>
> </blokquote>
>
>
> On Mon, Apr 29, 2013 at 10:21 AM, Kevin Burton < [EMAIL PROTECTED]>
> wrote:
> I looked at the link you provided and found the Ubuntu is one of the
> “supported platforms” but it doesn’t give any information on how to obtain
> it or build it. Any idea why it is not includde as part of the Cloudera CDH4
> distribution? I followed the installation instructions (mostly apt-get
> install . . . .) but I fail to see the libhadoop.so.  In order to avoid this
> warning do I need to download the Apache distribution? Which one?
>
> For the warnings about the configuration I looked in my configuration and
> for this specific example I don’t see ‘ session.id’ used anywhere. It must
> be used by default. If so why is the deprecated default being used?
>
> As for the two warnings about counters. I know I have not implemented any
> code for counters so again this must be something internal. Is there
> something I am doing to trigger this?
>
> So I can avoid them what are “hadoop generic options”?
>
> Thanks again.
>
> Kevin
>
> From: Ted Xu [mailto: [EMAIL PROTECTED]]
> Sent: Friday, April 26, 2013 10:49 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Warnings?
>
> Hi Kevin,
>
> Please see my comments inline,
>
> On Sat, Apr 27, 2013 at 11:24 AM, Kevin Burton < [EMAIL PROTECTED]>
> wrote:
> Is the native library not available for Ubuntu? If so how do I load it?
> Native libraries usually requires recompile, for more information please
> refer Native Libraries.
>
>
> Can I tell which key is off? Since I am just starting I would want to be as
> up to date as possible. It is out of date probably because I copied my
> examples from books and tutorials.
>
> I think the warning messages are telling it already, "xxx is deprecated, use
> xxx instead...". In fact, most of the configure keys are changed from hadoop
> 1.x to 2.x. The compatibility change may later documented on
> http://wiki.apache.org/hadoop/Compatibility.

Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB