Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> Hadoop metric are not collected on amazon EMR


+
meena k.s 2012-04-03, 11:10
+
Meena K.S 2012-04-03, 10:33
+
Eric Yang 2012-04-03, 16:06
+
meena k.s 2012-04-04, 08:03
+
Eric Yang 2012-04-05, 03:02
+
meena k.s 2012-04-04, 08:43
+
Ahmed Fathalla 2012-04-04, 11:09
+
meena k.s 2012-04-04, 11:11
+
Eric Yang 2012-04-05, 03:04
+
meena k.s 2012-04-05, 05:12
Copy link to this message
-
Re: Hadoop metric are not collected on amazon EMR
Hi Meena,

I can't pin point the exact problem, but there are two possible
scenarios leading to this issues.

1. Is there may be a mismatch of the hadoop jar file between your
Hadoop cluster and the hadoop jar file in
CHUKWA_HOME/share/chukwa/lib.  By default, Chukwa is shipped with
hadoop-core-1.0.0.jar.  You should replace hadoop-*.jar with the
0.20.205 jar files, then restart the collector.

2. Is your cluster running with Kerberos security turn on?  If yes,
make sure you have the proper core-site.xml in Chukwa classpath and
users keytabs setup.  kinit has been initialized for the Chukwa user.

regards,
Eric

On Wed, Apr 4, 2012 at 10:12 PM, meena k.s <[EMAIL PROTECTED]> wrote:
> Hi Eric,
>
> I had modified the chukwa-collector-conf.xml to stream to hdfs as mentioned
> in the link. but still i get the same error.
>
> I'm using chukwa 0.5 and hadoop version 0.20.205
>
> Thanks,
> Meena
>
>
> On Thu, Apr 5, 2012 at 8:34 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>
>> Which version of Hadoop are you using?  This looks like a mismatch
>> between how Chukwa
>> calls DFS client, and Hadoop jar file.
>>
>> regards,
>> Eric
>>
>> On Wed, Apr 4, 2012 at 1:43 AM, meena k.s <[EMAIL PROTECTED]>
>> wrote:
>> > Hi Eric,
>> >
>> > Regarding my earlier query, i see the following error message in
>> > chukwa-collector.log
>> >
>> >  WARN main SeqFileWriter - Got an exception trying to rotate. Will try
>> > again
>> > in 300 seconds.
>> > org.apache.hadoop.ipc.RemoteException: java.io.IOException:
>> > java.lang.NoSuchMethodException:
>> > org.apache.hadoop.hdfs.protocol.ClientProtocol.create(java.lang.String,
>> > org.apache.hadoop.fs.permission.FsPermission, java.lang.String, boolean,
>> > boolean, short, long)
>> >         at java.lang.Class.getMethod(Class.java:1605)
>> >         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:557)
>> >         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1388)
>> >         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1384)
>> >         at java.security.AccessController.doPrivileged(Native Method)
>> >         at javax.security.auth.Subject.doAs(Subject.java:396)
>> > Thanks,
>> > Meena
>> >
>> > On Wed, Apr 4, 2012 at 1:33 PM, meena k.s <[EMAIL PROTECTED]>
>> > wrote:
>> >>
>> >> Hi Eric,
>> >>
>> >> Thanks for the quick reply.I have tried to install chukwa 0.5
>> >> accordingly.
>> >> but there seems to be a problem while starting chukwa-collector. In the
>> >> log
>> >> files i could see something related to hbase. I haven't installed any
>> >> hbase
>> >> in my cluster. Therefore I commented out the parameters related to
>> >> hbase in
>> >> my chukwa-collector-conf.xml. Is it necessary to have a hbase
>> >> installation
>> >> for chukwa 0.5 to work.
>> >>
>> >> Thanks,
>> >> Meena
>> >>
>> >> On Tue, Apr 3, 2012 at 9:36 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> >>>
>> >>> Hi Meena,
>> >>>
>> >>> Hadoop 0.20.205 and Chukwa 0.4 is a mismatch.  Hadoop 0.20.20x has
>> >>> been modified to use MetricsSink instead of MentricsContext.
>> >>> Therefore, you will not see Hadoop metrics by using Chukwa 0.4.  You
>> >>> will need Chukwa 0.5 to monitor Hadoop 0.20.205+.  In addition, you
>> >>> should copy hadoop-metrics2.properties to $HADOOP_CONF_DIR to monitor
>> >>> Hadoop.  Hope this helps.
>> >>>
>> >>> regards,
>> >>> Eric
>> >>>
>> >>> On Tue, Apr 3, 2012 at 3:33 AM, Meena K.S <[EMAIL PROTECTED]>
>> >>> wrote:
>> >>> > Hi,
>> >>> >
>> >>> >
>> >>> >
>> >>> > We want to use chukwa for hadoop log collection. We are running hive
>> >>> > queries on EMR. So for the purpose of experimenting with chukwa we
>> >>> > configured chukwa on a 2 node EMR cluster.  The collector and the
>> >>> > agent are running and also the system logs are generated but the
>> >>> > hadoop mapred and rpc /jvm logs are not being generated. Chukwa
>> >>> > version : 0.4 and the  hadoop version :0.20.205
>> >>> >
>> >>> >
>> >>> >
>> >>> > My hadoop-metrics.properties is as below :
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB