Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> How to use TsProcessor ?


Copy link to this message
-
Re: How to use TsProcessor ?
> TsProcessor.time.regex.[some_data_type] maps to ColumnFamily

Eric, you lost me here. Why does the regular expression map to a column
family?
On Mon, Jun 13, 2011 at 9:38 AM, Eric Yang <[EMAIL PROTECTED]> wrote:

>  Hbase schema is defined by annotation in demux  parsers.  For
> TsProcessor, it is a generic parser, it is not targeting a specific data
> type.  HBaseWriter is currently not handling this generic parser well.
> The current implementation is writing data processed by TsProcessor to:
>
> TableName: TsProcessor
> ColumnFamily: log
>
> It may be possible to add handling of TsProcessor with this specification:
>
> Chunk DataType maps to TableName
> TsProcessor.time.regex.[some_data_type] maps to ColumnFamily
>
> If this is reasonable implementation, please file a jira.  Thanks
>
> Regards,
> Eric
>
>
> On 6/13/11 8:56 AM, "Bill Graham" <[EMAIL PROTECTED]> wrote:
>
> Apologies, but the documentation around processor configs is somewhat out
> dated (CHUKWA-538).
>
> For one, TsProcessor is not the default. DefaultProcessor is. You can
> change this with the chukwa.demux.mapper.default.processor setting.
>
> https://issues.apache.org/jira/browse/CHUKWA-473
>
> Also, ithe docs don't include some enhancements to TsProcessor to make it
> more multi-purpose. It can handle any data type passed to it as long as it
> can parse out the date. See this JIRA for how to override the default or the
> per-dataType date parsing logic:
>
> https://issues.apache.org/jira/browse/CHUKWA-472
>
>
> Eric is best to field the HBase schema question.
>
>
> On Mon, Jun 13, 2011 at 1:57 AM, DKN <[EMAIL PROTECTED]> wrote:
>
> I am just wondering if TsProcessor is a generic de-mux processor ? Is there
> a
> documentation for what is TsProcessor and if there is a customizable way of
> using it for generic data types defined ? I wanted to extend a demux
> processor for log processing and came to know that TsProcessor is default
> in
> this wiki link : http://wiki.apache.org/hadoop/DemuxModification
>
> I also couldn't see this table definition for HBase (in the hbase.schema).
> Where can I find the schema that I can create the table definitions in
> HBase
> ..
>
> Thanks in advance.
>
> Cheers, DKN
>
> --
> View this message in context:
> http://apache-chukwa.679492.n3.nabble.com/How-to-use-TsProcessor-tp3058006p3058006.html
> Sent from the Chukwa - Users mailing list archive at Nabble.com.
>
>
>
>