Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> How to use TsProcessor ?


Copy link to this message
-
Re: How to use TsProcessor ?
> TsProcessor.time.regex.[some_data_type] maps to ColumnFamily

Eric, you lost me here. Why does the regular expression map to a column
family?
On Mon, Jun 13, 2011 at 9:38 AM, Eric Yang <[EMAIL PROTECTED]> wrote:

>  Hbase schema is defined by annotation in demux  parsers.  For
> TsProcessor, it is a generic parser, it is not targeting a specific data
> type.  HBaseWriter is currently not handling this generic parser well.
> The current implementation is writing data processed by TsProcessor to:
>
> TableName: TsProcessor
> ColumnFamily: log
>
> It may be possible to add handling of TsProcessor with this specification:
>
> Chunk DataType maps to TableName
> TsProcessor.time.regex.[some_data_type] maps to ColumnFamily
>
> If this is reasonable implementation, please file a jira.  Thanks
>
> Regards,
> Eric
>
>
> On 6/13/11 8:56 AM, "Bill Graham" <[EMAIL PROTECTED]> wrote:
>
> Apologies, but the documentation around processor configs is somewhat out
> dated (CHUKWA-538).
>
> For one, TsProcessor is not the default. DefaultProcessor is. You can
> change this with the chukwa.demux.mapper.default.processor setting.
>
> https://issues.apache.org/jira/browse/CHUKWA-473
>
> Also, ithe docs don't include some enhancements to TsProcessor to make it
> more multi-purpose. It can handle any data type passed to it as long as it
> can parse out the date. See this JIRA for how to override the default or the
> per-dataType date parsing logic:
>
> https://issues.apache.org/jira/browse/CHUKWA-472
>
>
> Eric is best to field the HBase schema question.
>
>
> On Mon, Jun 13, 2011 at 1:57 AM, DKN <[EMAIL PROTECTED]> wrote:
>
> I am just wondering if TsProcessor is a generic de-mux processor ? Is there
> a
> documentation for what is TsProcessor and if there is a customizable way of
> using it for generic data types defined ? I wanted to extend a demux
> processor for log processing and came to know that TsProcessor is default
> in
> this wiki link : http://wiki.apache.org/hadoop/DemuxModification
>
> I also couldn't see this table definition for HBase (in the hbase.schema).
> Where can I find the schema that I can create the table definitions in
> HBase
> ..
>
> Thanks in advance.
>
> Cheers, DKN
>
> --
> View this message in context:
> http://apache-chukwa.679492.n3.nabble.com/How-to-use-TsProcessor-tp3058006p3058006.html
> Sent from the Chukwa - Users mailing list archive at Nabble.com.
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB