Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - HDFS custom escape sequence


+
Camp, Roy 2013-06-10, 21:51
Copy link to this message
-
RE: HDFS custom escape sequence
Paul Chavez 2013-06-10, 22:05
Yes, any header works using that same notation. I use it to allow us to bucket many event 'types' with a single channel.

for example, a path with custom headers and Hive ready partition folders:
/flumelogs/%{someHeader}/%{someOtherHeader}/datekey=%Y%m%d/hour=%H

Hope that helps,
Paul Chavez

________________________________
From: Camp, Roy [mailto:[EMAIL PROTECTED]]
Sent: Monday, June 10, 2013 2:51 PM
To: [EMAIL PROTECTED]
Subject: HDFS custom escape sequence

Is it possible to use other headers besides the host header, %{host} escape sequence, to name the HDFS files?  We are interested in attaching a category header and want the events stored separately.

If not possible, is there a best practice for achieving this?

Thanks,

Roy
+
Hari Shreedharan 2013-06-10, 22:04