Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Re: is there a concept similar to scribe's category in flume-ng


Copy link to this message
-
Re: is there a concept similar to scribe's category in flume-ng
Arvind Prabhakar 2012-12-26, 21:29
[Moving the conversation to [EMAIL PROTECTED] as this is not a CDH
specific question]

Flume allows writing events to a directory whose path contains
substitutable tokens. At the time of writing out these events, these tokens
are substituted with corresponding header values. You can provide these
header values at the time these events are written to the Flume pipleline,
or use appropriate interceptors to add these headers anywhere in the
pipeline as convenient.

For more information on this please see documentation for HDFS sink [1] and
static interceptor [2]. Of course there are more sophisticated ways of
achieving this functionality that offer other advantages, but this should
be sufficient to get you scribe like functionality that you are asking
about.

[1] http://flume.apache.org/FlumeUserGuide.html#hdfs-sink
[2] http://flume.apache.org/FlumeUserGuide.html#static-interceptor

Regards,
Arvind

On Wed, Dec 26, 2012 at 12:39 AM, panfei <[EMAIL PROTECTED]> wrote:

> I found this:
> http://archive.cloudera.com/cdh4/cdh/4/flume-ng/FlumeUserGuide.html#scribe-source
>
>
> 2012/12/26 panfei <[EMAIL PROTECTED]>
>
>> when gathering data from the web service, we send our data to a
>> specified  category to scribe, and the data will be put exactly under a
>> directory named the  category name.
>>
>>
>> if I want to migrate from scribe to flume , we need the same effect ? so
>> we need something like category in flume, any tips ?
>>
>> thanks very much.
>>
>> --
>> 不学习,不知道
>>
>
>
>
> --
> 不学习,不知道
>
> --
>
>
>
>