Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Could not instantiate class org.apache.flume.clients.log4jappender


+
孙东亮 2013-01-14, 03:25
+
Chhaya Vishwakarma 2013-01-14, 04:52
+
Dongliang Sun 2013-01-14, 05:24
+
Connor Woodson 2013-01-14, 05:50
+
Dongliang Sun 2013-01-14, 06:05
Copy link to this message
-
RE: Could not instantiate class org.apache.flume.clients.log4jappender
Hi,
This error is coming because it needs some other jar from FLUME_HOME/lib/ I can’t tell you name of specific jar but I was facing same problem.Put jetty,Jackson,avro,slf4 and some more from  FLUME_HOME/lib/ because every time you will run your script  will give you different error like java.lang.NoClassDefFoundError

From: Dongliang Sun [mailto:[EMAIL PROTECTED]]
Sent: Monday, January 14, 2013 11:36 AM
To: [EMAIL PROTECTED]; Chhaya Vishwakarma; [EMAIL PROTECTED]
Subject: Re: Could not instantiate class org.apache.flume.clients.log4jappender

REGISTER and -Dpig.additional.libraries both do not work, same error, could not find the class.
If put the particular jars in pig's lib, the following error:
Error before Pig is launched
----------------------------
ERROR 2998: Unhandled internal error. org/jboss/netty/channel/ChannelFactory

java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
        at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
        at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
        at org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)
        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
        at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
        at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
        at org.apache.pig.Main.configureLog4J(Main.java:678)
        at org.apache.pig.Main.run(Main.java:337)
        at org.apache.pig.Main.main(Main.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        ... 19 more

2013/1/14 Connor Woodson <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
You could try calling 'REGISTER <log4jappender jar>' at the start of your script, or on the command line by adding -Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't know if that will work or not though.

You could try placing it in pig's lib folder.

You also need to include the flume-ng-sdk jar the same way as the log4jappender one, as the sdk is a dependency of the appender.

On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
Whether there is some problem in my PIG log4j.properties?
Do you have any experience using log4jappender on PIG?
Thanks,
Dongliang
2013/1/14 Chhaya Vishwakarma <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>

Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar in CLASSPATH. Jar can be found in FLUME_HOME/lib/

From: 孙东亮 [mailto:[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>]
Sent: Monday, January 14, 2013 8:55 AM
To: [EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>
Subject: Could not instantiate class org.apache.flume.clients.log4jappender

I'm a newbie for the Flume, and I just set up flume for a test.
I want to use the log4jappender to get log info from PIG scripts, the log4j.properties is:

log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender

log4j.appender.flume.Hostname = localhost

log4j.appender.flume.Port = 41414

log4j.logger.org.apache.pig=DEBUG,flume

And for the flume.config:

a1.sources = r1

a1.sinks = k1

a1.channels = c1

# Describe/configure the source

a1.sources.r1.type = avro

a1.sources.r1.bind = localhost

a1.sources.r1.port = 41414

# Describe the sink

a1.sinks.k1.type = logger

# Use a channel which buffers events in memory

a1.channels.c1.type = memory

a1.channels.c1.capacity = 1000

a1.channels.c1.transactionCapacity = 100

# Bind the source and sink to the channel

a1.sources.r1.channels = c1

a1.sinks.k1.channel = c1

But I got the following error when run a pig script:
java.lang.ClassNotFoundException: org.apache.flume.clients.log4jappender.Log4jAppender
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:169)
at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179)
at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320)
at org.apache.log4j.helpers
+
Alexander Alten-Lorenz 2013-01-14, 06:58
+
Connor Woodson 2013-01-14, 07:10
+
Alexander Alten-Lorenz 2013-01-14, 08:53
+
Dongliang Sun 2013-01-14, 09:30
+
Alexander Alten-Lorenz 2013-01-14, 10:01
+
Dongliang Sun 2013-01-14, 07:11
+
Alexander Alten-Lorenz 2013-01-14, 08:13
+
Dongliang Sun 2013-01-14, 08:37
+
Dongliang Sun 2013-01-14, 05:42
+
Dongliang Sun 2013-01-14, 06:24
+
Chhaya Vishwakarma 2013-01-14, 07:09
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB