Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Warnings?


+
rkevinburton@... 2013-04-26, 21:51
+
Ted Xu 2013-04-27, 00:49
Copy link to this message
-
Re: Warnings?
rkevinburton@... 2013-04-29, 19:22

I don't have this environment  variable. Should I create it in .bashrc
AND /etc/profile?

On Mon, Apr 29, 2013 at 1:55 PM, Omkar Joshi wrote:

  Hi,

did you check in your ubuntu installation; "libhadoop" binary.. it is
present in my ubuntu installation at a relative path of (I used apache
installation)

"hadoop-common-project/hadoop-common/target/native/target/usr/local/lib"

if present add it to your LID_LIBRARY_PATH.

if not present then you can try rebuilding your hadoop installation

"mvn clean install -Pnative -Pdist -Dtar -DskipTests"
Thanks, Omkar Joshi
Hortonworks Inc. <http://www.hortonworks.com>

On Mon, Apr 29, 2013 at 11:19 AM, Kevin Burton <
[EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
> wrote:
If it doesn't work what are my options? Is there source that I can
download and compile?
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

On Apr 29, 2013, at 10:31 AM, Ted Xu < [EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')> >
wrote:

  <javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Hi Kevin,
  <javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Native libraries are those implemented using C/C++, which only provide
code level portability (instead of binary level portability, as Java
do). That is to say, the binaries provided by CDH4 distribution will in
most cases be broken in your environment. 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

  <javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
To check if your native libraries are working or not, you can follow the
instructions I sent previously. Quote as following.
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

  <javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
<blockquote>
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
During runtime, check the hadoop log files for your MapReduce tasks.
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

    • If everything is all right, then:  DEBUG util.NativeCodeLoader -
Trying to load the custom-built native-hadoop library...   INFO
util.NativeCodeLoader - Loaded the native-hadoop library
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
    • If something goes wrong, then:  INFO util.NativeCodeLoader -
Unable to load native-hadoop library for your platform... using
builtin-java classes where applicable
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

</blokquote>
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>

On Mon, Apr 29, 2013 at 10:21 AM, Kevin Burton <
[EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
> wrote:
I looked at the link you provided and found the Ubuntu is one of the
“supported platforms” but it doesn’t give any information on how to
obtain it or build it. Any idea why it is not includde as part of the
Cloudera CDH4 distribution? I followed the installation instructions
(mostly apt-get install . . . .) but I fail to see the libhadoop.so.  In
order to avoid this warning do I need to download the Apache
distribution? Which one?
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
For the warnings about the configuration I looked in my configuration
and for this specific example I don’t see ‘ session.id
<http://session.id> ’ used anywhere. It must be used by default. If so
why is the deprecated default being used?  <http://session.id>
   <http://session.id>
As for the two warnings about counters. I know I have not implemented
any code for counters so again this must be something internal. Is there
something I am doing to trigger this?  <http://session.id>
   <http://session.id>
So I can avoid them what are “hadoop generic options”?
<http://session.id>
   <http://session.id>
Thanks again.  <http://session.id>
   <http://session.id>
Kevin  <http://session.id>
   <http://session.id>
From: Ted Xu [mailto: [EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')> ]
Sent: Friday, April 26, 2013 10:49 PM
To: [EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Subject: Re: Warnings?
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Hi Kevin,
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Please see my comments inline,
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
 
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
On Sat, Apr 27, 2013 at 11:24 AM, Kevin Burton <
[EMAIL PROTECTED]
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Is the native library not available for Ubuntu? If so how do I load it?
<javascript:parent.wgMail.openComposeWindow('[EMAIL PROTECTED]')>
Native libraries usually requires recompile, for more information please
refer Native Libraries
<http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/NativeLibraries.html>

<http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/NativeLibraries.html>
 
<http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/NativeLibraries.html>
 
<http://hadoop.apache.org/docs/r2.0.4-alpha/hadoop-project-dist/hadoop-common/NativeLibraries.html>
Can I tell which key is off? Since I am just starting I would want to be
as up to date as possible. It is out of date probably because I copied
my examples from books and tutorials.
<http://hadoop.apache.org/docs/r2.0.4-al
+
Kevin Burton 2013-04-29, 18:19