Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> Writing logs collected by chukwa to hbase


+
NehaS Singh 2012-12-28, 10:56
+
Eric Yang 2013-01-05, 18:25
+
NehaS Singh 2013-01-06, 14:00
+
Eric Yang 2013-01-06, 21:17
+
Shouvanik Haldar 2013-01-07, 09:17
+
Eric Yang 2013-01-08, 04:52
Copy link to this message
-
Re: Writing logs collected by chukwa to hbase
Hi,

I was following the post -> http://www.undercloud.org/?p=429. I did the
steps as was mentioned here. But, when I was going to check in HDFS whether
data has come from chukwa, I cannot see the page. I get the exception as

Also, I want to know what is the default location of chukwa logs. I found
out a location ->
"/home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/logs". Is this the
location? Previously, I had some logs inside this directory. I have deleted
these logs. Now, no logs are getting created.

Previously, when logs were existing, I checked one log entry. It says
"nice: /home/ec2-user/chukwa/chukwa-incubating-0.5.0/sbin/bin/chukwa: No
such file or directory".
I have edited bash_profile to give the path of CHUKWA_HOME. Is this
required?

I am using "chukwa-incubating-0.5.0" and "hadoop-1.0.4". I can see data
getting collected at the collector by browsing the url =>
http://ec2-54-234-57-44.compute-1.amazonaws.com:8080/chukwa?ping=true

But somehow, data not getting inside hdfs. Please help.
Regards,
Shouvanik
On Tue, Jan 8, 2013 at 10:22 AM, Eric Yang <[EMAIL PROTECTED]> wrote:

> The quick start guide may help:
>
> http://incubator.apache.org/chukwa/docs/r0.5.0/Quick_Start_Guide.html
>
> regards,
> Eric
>
>
> On Mon, Jan 7, 2013 at 1:17 AM, Shouvanik Haldar <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>> I want to install chukwa on a single node and test it. Where can I find a
>> suitable document?
>>
>>
>> On Mon, Jan 7, 2013 at 2:47 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>
>>> There are a few things that you need to do:
>>>
>>> 1. Configure the collector to use HBaseWriter.
>>>
>>>   http://incubator.apache.org/chukwa/docs/r0.5.0/collector.html
>>>
>>> Make sure that hadoop and hbase configuration are in the classpath of
>>> the collector.
>>>
>>> 2. Write a demux parser for your log file to store on HBase, and this
>>> runs in chukwa collector.
>>>
>>>   http://incubator.apache.org/chukwa/docs/r0.5.0/programming.html
>>>
>>> 3. What log4j adaptors and appenders are you using to write the log
>>> files?
>>>   I would recommend to use socket appender from Log4j and use socket
>>> adaptor to listen for the incoming data stream.  This will ensure the
>>> records are delimited properly.
>>>
>>> Hope this helps.
>>>
>>> regards,
>>> Eric
>>>
>>> On Sun, Jan 6, 2013 at 6:00 AM, NehaS Singh <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>> I am using chukwa 0.5. My probleam is want to store the logs from log4j
>>>> or other party which is not hadoop.Also i want to  write the logs to hbase.
>>>> But i am not ale to do so. I have configuerd log4j.properties file so i
>>>> am able to see th logs in hdfs.
>>>> what are the configuration or steps using which i can write the logs to
>>>> hbase.
>>>> Please help.
>>>> ________________________________________
>>>> From: Eric Yang [[EMAIL PROTECTED]]
>>>> Sent: Saturday, January 05, 2013 11:55 PM
>>>> To: [EMAIL PROTECTED]
>>>> Subject: Re: Writing logs collected by chukwa to hbase
>>>>
>>>> What version of Chukwa are you running?
>>>>
>>>> regards,
>>>> Eric
>>>>
>>>> On Fri, Dec 28, 2012 at 2:56 AM, NehaS Singh <
>>>> [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
>>>> Hi,
>>>>                 I am using Chukwa to collect log4j logs which I can see
>>>> in HDFS. But I am not able to write it to HBASE. Can anyone help ?
>>>>
>>>> Regards,
>>>> Neha Singh
>>>>
>>>>
>>>> ________________________________
>>>> The contents of this e-mail and any attachment(s) may contain
>>>> confidential or privileged information for the intended recipient(s).
>>>> Unintended recipients are prohibited from taking action on the basis of
>>>> information in this e-mail and using or disseminating the information, and
>>>> must notify the sender and delete it from their system. L&T Infotech will
>>>> not accept responsibility or liability for the accuracy or completeness of,
>>>> or the presence of any virus or disabling code in this e-mail"
>

Thanks,
*Shouvanik*
+
Shouvanik Haldar 2013-01-08, 12:52
+
Shouvanik Haldar 2013-01-08, 13:04
+
Shouvanik Haldar 2013-01-08, 14:50
+
NehaS Singh 2013-01-07, 06:18
+
Eric Yang 2013-01-08, 04:51
+
Sreepathi 2012-12-28, 19:22