Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> Writing logs collected by chukwa to hbase

Copy link to this message
Re: Writing logs collected by chukwa to hbase

I was following the post -> http://www.undercloud.org/?p=429. I did the
steps as was mentioned here. But, when I was going to check in HDFS whether
data has come from chukwa, I cannot see the page. I get the exception as

Also, I want to know what is the default location of chukwa logs. I found
out a location ->
"/home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/logs". Is this the
location? Previously, I had some logs inside this directory. I have deleted
these logs. Now, no logs are getting created.

Previously, when logs were existing, I checked one log entry. It says
"nice: /home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/bin/chukwa: No
such file or directory".
I have edited bash_profile to give the path of CHUKWA_HOME. Is this

I am using "chukwa-incubating-0.5.0" and "hadoop-1.0.4". I can see data
getting collected at the collector by browsing the url =>

But somehow, data not getting inside hdfs. Please help.
On Tue, Jan 8, 2013 at 10:22 AM, Eric Yang <[EMAIL PROTECTED]> wrote:

> The quick start guide may help:
> http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html
> regards,
> Eric
> On Mon, Jan 7, 2013 at 1:17 AM, Shouvanik Haldar <
>> Hi,
>> I want to install chukwa on a single node and test it. Where can I find a
>> suitable document?
>> On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>> There are a few things that you need to do:
>>> 1. Configure the collector to use HBaseWriter.
>>>   http://incubator.apache.org/chukwa/docs/r0.5.0/collector.html
>>> Make sure that hadoop and hbase configuration are in the classpath of
>>> the collector.
>>> 2. Write a demux parser for your log file to store on HBase, and this
>>> runs in chukwa collector.
>>>   http://incubator.apache.org/chukwa/docs/r0.5.0/programming.html
>>> 3. What log4j adaptors and appenders are you using to write the log
>>> files?
>>>   I would recommend to use socket appender from Log4j and use socket
>>> adaptor to listen for the incoming data stream.  This will ensure the
>>> records are delimited properly.
>>> Hope this helps.
>>> regards,
>>> Eric
>>> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <[EMAIL PROTECTED]
>>> > wrote:
>>>> I am using chukwa 0.5. My probleam is want to store the logs from log4j
>>>> or other party which is not hadoop.Also i want to  write the logs to hbase.
>>>> But i am not ale to do so. I have configuerd log4j.properties file so i
>>>> am able to see th logs in hdfs.
>>>> what are the configuration or steps using which i can write the logs to
>>>> hbase.
>>>> Please help.
>>>> ________________________________________
>>>> From: Eric Yang [[EMAIL PROTECTED]]
>>>> Sent: Saturday, January 05, 2013 11:55 PM
>>>> Subject: Re: Writing logs collected by chukwa to hbase
>>>> What version of Chukwa are you running?
>>>> regards,
>>>> Eric
>>>> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <
>>>> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
>>>> Hi,
>>>>                 I am using Chukwa to collect log4j logs which I can see
>>>> in HDFS. But I am not able to write it to HBASE. Can anyone help ?
>>>> Regards,
>>>> Neha Singh
>>>> ________________________________
>>>> The contents of this e-mail and any attachment(s) may contain
>>>> confidential or privileged information for the intended recipient(s).
>>>> Unintended recipients are prohibited from taking action on the basis of
>>>> information in this e-mail and using or disseminating the information, and
>>>> must notify the sender and delete it from their system. L&T Infotech will
>>>> not accept responsibility or liability for the accuracy or completeness of,
>>>> or the presence of any virus or disabling code in this e-mail"