Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Could not instantiate class org.apache.flume.clients.log4jappender


Copy link to this message
-
RE: Could not instantiate class org.apache.flume.clients.log4jappender
Which version of slf4j you using ? for this error you need to put SLF4J 1.7.2 also in classpath as org.slf4j.spi.LocationAwareLogger.log is available in 1.7.2 of slf4j.

From: Dongliang Sun [mailto:[EMAIL PROTECTED]]
Sent: Monday, January 14, 2013 11:54 AM
To: Connor Woodson; Chhaya Vishwakarma; [EMAIL PROTECTED]
Subject: Re: Could not instantiate class org.apache.flume.clients.log4jappender

It does not work, -Dpig.additional.jars={REALPATH}/lib/*.
Also tried Chhaya said, I put all the libraries from FLUME_HOME/lib/* to PIG_HOME/lib/, got following error:

13/01/14 14:19:03 WARN api.NettyAvroRpcClient: Invalid value for batchSize: 0; Using default value.
Exception in thread "main" java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V
    at org.apache.commons.logging.impl.SLF4JLocationAwareLog.error(SLF4JLocationAwareLog.java:211)
    at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:125)
    at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:79)
    at org.apache.pig.Main.run(Main.java:589)
    at org.apache.pig.Main.main(Main.java:111)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:597)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

2013/1/14 Connor Woodson <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Alright so it is just before you weren't adding them properly to the classpath. I am not quite sure how to do that; a look at Pig's run script (bin/pig) might yield insight into an option to add them. Putting the files in pig's lib's folder almost works; the issue is those libraries have more dependencies that aren't in the classpath yet. Looking at their (log4jappender and sdk) poms could tell you exactly which libs you need to include, but let's not do that.

I mistyped the -D command from before. Try this:

-Dpig.additional.jars=${FLUME_HOME}/lib/*

FLUME_HOME obviously needs to be declared beforehand, or just replace that with the real path (and make sure the log4jappender and sdk jars, and all other jars, are in the libs folder)

On Sun, Jan 13, 2013 at 10:05 PM, Dongliang Sun <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
REGISTER and -Dpig.additional.libraries both do not work, same error, could not find the class.
If put the particular jars in pig's lib, the following error:
Error before Pig is launched
----------------------------
ERROR 2998: Unhandled internal error. org/jboss/netty/channel/ChannelFactory

java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
        at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
        at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
        at org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)
        at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133)
        at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97)
        at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689)

        at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647)
        at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568)
        at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442)
        at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367)
        at org.apache.pig.Main.configureLog4J(Main.java:678)
        at org.apache.pig.Main.run(Main.java:337)
        at org.apache.pig.Main.main(Main.java:111)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory

        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        ... 19 more

2013/1/14 Connor Woodson <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
You could try calling 'REGISTER <log4jappender jar>' at the start of your script, or on the command line by adding -Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't know if that will work or not though.

You could try placing it in pig's lib folder.

You also need to include the flume-ng-sdk jar the same way as the log4jappender one, as the sdk is a dependency of the appender.

On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Actually I have append the FLUME_HOME/lib/ to my CLASSPATH.
Whether there is some problem in my PIG log4j.properties?
Do you have any experience using log4jappender on PIG?
Thanks,
Dongliang
2013/1/14 Chhaya Vishwakarma <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>

Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.ja