Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> Writing logs collected by chukwa to hbase

Copy link to this message
Re: Writing logs collected by chukwa to hbase
I want to install chukwa on a single node and test it. Where can I find a
suitable document?
On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang <[EMAIL PROTECTED]> wrote:

> There are a few things that you need to do:
> 1. Configure the collector to use HBaseWriter.
>   http://incubator.apache.org/chukwa/docs/r0.5.0/collector.html
> Make sure that hadoop and hbase configuration are in the classpath of the
> collector.
> 2. Write a demux parser for your log file to store on HBase, and this runs
> in chukwa collector.
>   http://incubator.apache.org/chukwa/docs/r0.5.0/programming.html
> 3. What log4j adaptors and appenders are you using to write the log files?
>   I would recommend to use socket appender from Log4j and use socket
> adaptor to listen for the incoming data stream.  This will ensure the
> records are delimited properly.
> Hope this helps.
> regards,
> Eric
> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <[EMAIL PROTECTED]>wrote:
>> I am using chukwa 0.5. My probleam is want to store the logs from log4j
>> or other party which is not hadoop.Also i want to  write the logs to hbase.
>> But i am not ale to do so. I have configuerd log4j.properties file so i
>> am able to see th logs in hdfs.
>> what are the configuration or steps using which i can write the logs to
>> hbase.
>> Please help.
>> ________________________________________
>> From: Eric Yang [[EMAIL PROTECTED]]
>> Sent: Saturday, January 05, 2013 11:55 PM
>> Subject: Re: Writing logs collected by chukwa to hbase
>> What version of Chukwa are you running?
>> regards,
>> Eric
>> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <[EMAIL PROTECTED]
>> <mailto:[EMAIL PROTECTED]>> wrote:
>> Hi,
>>                 I am using Chukwa to collect log4j logs which I can see
>> in HDFS. But I am not able to write it to HBASE. Can anyone help ?
>> Regards,
>> Neha Singh
>> ________________________________
>> The contents of this e-mail and any attachment(s) may contain
>> confidential or privileged information for the intended recipient(s).
>> Unintended recipients are prohibited from taking action on the basis of
>> information in this e-mail and using or disseminating the information, and
>> must notify the sender and delete it from their system. L&T Infotech will
>> not accept responsibility or liability for the accuracy or completeness of,
>> or the presence of any virus or disabling code in this e-mail"