Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Possible to use Regex Interceptor in Flume 1.2


Copy link to this message
-
RE: Possible to use Regex Interceptor in Flume 1.2
Hello,

Finally got around to testing this...

I have built and repackaged a JAR with the following classes:

/usr/lib/flume-ng/lib$ jar tf flume-ng-RegexCaptureInterceptor-1.3.jar
META-INF/
META-INF/MANIFEST.MF
META-INF/DEPENDENCIES
META-INF/LICENSE
META-INF/maven/
META-INF/maven/org.apache.flume/
META-INF/maven/org.apache.flume/flume-ng-core/
META-INF/maven/org.apache.flume/flume-ng-core/pom.properties
META-INF/maven/org.apache.flume/flume-ng-core/pom.xml
META-INF/NOTICE
org/
org/apache/
org/apache/flume/
org/apache/flume/interceptor/
org/apache/flume/interceptor/RegexExtractorInterceptor$1.class
org/apache/flume/interceptor/RegexExtractorInterceptor$Builder.class
org/apache/flume/interceptor/RegexExtractorInterceptor$NameAndSerializer.class
org/apache/flume/interceptor/RegexExtractorInterceptor.class
org/apache/flume/interceptor/RegexExtractorInterceptorMillisSerializer.class
org/apache/flume/interceptor/RegexExtractorInterceptorPassThroughSerializer.class
org/apache/flume/interceptor/RegexExtractorInterceptorSerializer.class

However when trying to start an agent with this setup I get the following errors:

4:55:16.866 PM ERROR org.apache.flume.channel.ChannelProcessor
Could not instantiate Builder. Exception follows.
java.lang.InstantiationException: org.apache.flume.interceptor.RegexExtractorInterceptor
at java.lang.Class.newInstance0(Class.java:340)
at java.lang.Class.newInstance(Class.java:308)
at org.apache.flume.interceptor.InterceptorBuilderFactory.newInstance(InterceptorBuilderFactory.java:48)
at org.apache.flume.channel.ChannelProcessor.configureInterceptors(ChannelProcessor.java:109)
at org.apache.flume.channel.ChannelProcessor.configure(ChannelProcessor.java:80)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)

4:55:16.867 PM ERROR org.apache.flume.conf.properties.PropertiesFileConfigurationProvider
Failed to load configuration data. Exception follows.
org.apache.flume.FlumeException: Interceptor.Builder not constructable.
at org.apache.flume.channel.ChannelProcessor.configureInterceptors(ChannelProcessor.java:117)
at org.apache.flume.channel.ChannelProcessor.configure(ChannelProcessor.java:80)
at org.apache.flume.conf.Configurables.configure(Configurables.java:41)

Any insight as to what's going wrong is appreciated.

Thank you,
Paul Chavez
-----Original Message-----
From: Brock Noland [mailto:[EMAIL PROTECTED]]
Sent: Monday, December 10, 2012 8:55 AM
To: [EMAIL PROTECTED]
Subject: Re: Possible to use Regex Interceptor in Flume 1.2

Hi,

If you built flume 1.3.0 and took the RegexExtractorInterceptor class files, built your own jar with just these class files, and placed that jar in Flumes classpath you'd be able to specify the fully qualified class name of the RegexExtractorInterceptor.

Too bad we did't make RegexExtractorInterceptor it's own module as it would be easier.

Brock

On Mon, Dec 10, 2012 at 10:36 AM, Paul Chavez <[EMAIL PROTECTED]> wrote:
> I would like to use the new regex interceptor to pull timestamp values out of my event data. However, I do not manage the Hadoop installation at my work and we are using the CDH distributions which currently have Flume 1.2 packaged in them.
>
> Is there a straightforward way to use just the regex interceptor with Flume 1.2 as packaged by CDH 4.x?
>
> thank you,
> Paul Chavez

--
Apache MRUnit - Unit testing MapReduce - http://incubator.apache.org/mrunit/