-Re: Writing logs collected by chukwa to hbase
Shouvanik Haldar 2013-01-08, 12:50
I was following the post -> http://www.undercloud.org/?p=429. I did the
steps as was mentioned here. But, when I was going to check in HDFS whether
data has come from chukwa, I cannot see the page. I get the exception as
Also, I want to know what is the default location of chukwa logs. I found
out a location ->
"/home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/logs". Is this the
location? Previously, I had some logs inside this directory. I have deleted
these logs. Now, no logs are getting created.
Previously, when logs were existing, I checked one log entry. It says
"nice: /home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/bin/chukwa: No
such file or directory".
I have edited bash_profile to give the path of CHUKWA_HOME. Is this
I am using "chukwa-incubating-0.5.0" and "hadoop-1.0.4". I can see data
getting collected at the collector by browsing the url =>
But somehow, data not getting inside hdfs. Please help.
On Tue, Jan 8, 2013 at 10:22 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
> The quick start guide may help:
> On Mon, Jan 7, 2013 at 1:17 AM, Shouvanik Haldar <
> [EMAIL PROTECTED]> wrote:
>> I want to install chukwa on a single node and test it. Where can I find a
>> suitable document?
>> On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>> There are a few things that you need to do:
>>> 1. Configure the collector to use HBaseWriter.
>>> Make sure that hadoop and hbase configuration are in the classpath of
>>> the collector.
>>> 2. Write a demux parser for your log file to store on HBase, and this
>>> runs in chukwa collector.
>>> 3. What log4j adaptors and appenders are you using to write the log
>>> I would recommend to use socket appender from Log4j and use socket
>>> adaptor to listen for the incoming data stream. This will ensure the
>>> records are delimited properly.
>>> Hope this helps.
>>> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <[EMAIL PROTECTED]
>>> > wrote:
>>>> I am using chukwa 0.5. My probleam is want to store the logs from log4j
>>>> or other party which is not hadoop.Also i want to write the logs to hbase.
>>>> But i am not ale to do so. I have configuerd log4j.properties file so i
>>>> am able to see th logs in hdfs.
>>>> what are the configuration or steps using which i can write the logs to
>>>> Please help.
>>>> From: Eric Yang [[EMAIL PROTECTED]]
>>>> Sent: Saturday, January 05, 2013 11:55 PM
>>>> To: [EMAIL PROTECTED]
>>>> Subject: Re: Writing logs collected by chukwa to hbase
>>>> What version of Chukwa are you running?
>>>> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <
>>>> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
>>>> I am using Chukwa to collect log4j logs which I can see
>>>> in HDFS. But I am not able to write it to HBASE. Can anyone help ?
>>>> Neha Singh
>>>> The contents of this e-mail and any attachment(s) may contain
>>>> confidential or privileged information for the intended recipient(s).
>>>> Unintended recipients are prohibited from taking action on the basis of
>>>> information in this e-mail and using or disseminating the information, and
>>>> must notify the sender and delete it from their system. L&T Infotech will
>>>> not accept responsibility or liability for the accuracy or completeness of,
>>>> or the presence of any virus or disabling code in this e-mail"