Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> hbase as sink


Hi

host1.sources = src1
host1.sinks = sink1
host1.channels = ch1

host1.sources.src1.type = exec
host1.sources.src1.command = tail -F /home/user/Flume/flume-ng/csa-node1.log
host1.sources.src1.channels = ch1
host1.sources.src1.restart = true

host1.sinks.sink1.type = org.apache.flume.sink.hbase.HBaseSink
host1.sinks.sink1.channel = ch1
host1.sinks.sink1.table = test10
host1.sinks.sink1.columnFamily = testing

host1.sinks.sink1.serializer = org.apache.flume.sink.hbase.RegexHbaseEventSerializer
host1.sinks.sink1.serializer.regex =(\.\w+)(?<=Exception)
host1.sinks.sink1.serializer.regexIgnoreCase = true
host1.sinks.sink1.serializer.colNames = exception

host1.channels.ch1.type=memory
Regards,
Chhaya Vishwakarma
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Thursday, December 26, 2013 9:41 PM
To: [EMAIL PROTECTED]
Subject: Re: hbase as sink

Please share your sink configuration.

On Thu, Dec 26, 2013 at 4:05 AM, Chhaya Vishwakarma <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi,

I want to parse a log file to get the highlighted text.

Regular expression used to parse

(\.\w+)(?<=Exception)

But table in hbase shows no rows
The above Regex  is working on the below sample data in regex editor (gskinner)

13 Aug 2013 11:28:30,414 [WebContainer : 6] ERROR - An Error has occured for com.marsh.framework.core.exception.MarshException: Your session has timed out.
13 Aug 2013 11:28:30,414 [WebContainer : 6] ERROR - handleException():com.marsh.framework.core.exception.MarshException: Your session has timed out.
13 Aug 2013 15:45:12,061 [WebContainer : 2] ERROR - Remote Exception while updating CSA Details java.rmi.ServerException: RemoteException occurred in server thread;nested exception is:

What notation does flume follows for regex? (for e.g pearl notation)

Regards,
Chhaya Vishwakarma
________________________________
The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"

--
Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB