Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - what are the libraries needed for flume log4jappender


Copy link to this message
-
Re: what are the libraries needed for flume log4jappender
Arvind Prabhakar 2013-01-23, 21:17
Here are the dependencies that maven shows for Flume's log4j appender
(generated from the 1.3.0 branch):

org.apache.flume.flume-ng-clients:flume-ng-log4jappender:jar:1.3.1:

   - log4j:log4j:jar:1.2.16
   - org.apache.flume:flume-ng-sdk:jar:1.3.1
      - org.apache.avro:avro:jar:1.7.2
         - org.codehaus.jackson:jackson-core-asl:jar:1.9.3
         - org.codehaus.jackson:jackson-mapper-asl:jar:1.9.3
         - com.thoughtworks.paranamer:paranamer:jar:2.3
         - org.xerial.snappy:snappy-java:jar:1.0.4.1
         - org.slf4j:slf4j-api:jar:1.6.1
      -  org.apache.avro:avro-ipc:jar:1.7.2
         - org.mortbay.jetty:jetty:jar:6.1.26
         - org.mortbay.jetty:jetty-util:jar:6.1.26
         - org.apache.velocity:velocity:jar:1.7
            - commons-collections:commons-collections:jar:3.2.1
            - commons-lang:commons-lang:jar:2.5
         - io.netty:netty:jar:3.4.0.Final
Regards,
Arvind Prabhakar

On Wed, Jan 23, 2013 at 12:44 PM, Mike Percy <[EMAIL PROTECTED]> wrote:

> If you want only the required library files, use Maven to pull in the
> exact set of dependencies.
>
>
> On Wed, Jan 23, 2013 at 12:24 PM, yogi nerella <[EMAIL PROTECTED]>wrote:
>
>> Mike,
>>
>> When I include all the libraries it is not giving compilation problem,
>> but log4jappender documentation says it only needs flume-ng-sdk.jar file.
>>
>> In that case, if I want to ship flume log4jappender, should I have to
>> ship all these jar files in flume/lib directory?
>>
>> Yogi
>>
>>
>> On Wed, Jan 23, 2013 at 12:08 PM, Mike Percy <[EMAIL PROTECTED]> wrote:
>>
>>> I don't use Eclipse but my understanding is that mvn eclipse:eclipse
>>> will create a usable Eclipse project file which you can import. the
>>> AvroFlumeEvent class is automatically generated at build time by the maven
>>> avro compiler.
>>>
>>> If you want to avoid maven entirely, you have the option of pulling in
>>> all the jar files in the lib directory of the Flume binary archive. Those
>>> are all the dependencies Flume needs to run.
>>>
>>> Hope that helps!
>>>
>>>
>>>
>>> On Wed, Jan 23, 2013 at 12:03 PM, yogi nerella <[EMAIL PROTECTED]>wrote:
>>>
>>>> Mike,
>>>>
>>>> I am importing the project into eclipse using maven, but the project
>>>> has issues in the legacy-sources module.
>>>>
>>>> Description Resource Path Location Type
>>>> AvroFlumeEvent cannot be resolved to a type NettyAvroRpcClient.java
>>>> /flume-ng-sdk/src/main/java/org/apache/flume/api line 189 Java Problem
>>>>
>>>>
>>>> Runtime issues are with 1.3.1
>>>> 1.   If I include just ng-core, I run into issues of FlumeException not
>>>> found.
>>>>
>>>> Classpath:
>>>> C:\ServiceManager\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-core-1.3.1.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-log4jappender-1.3.1.jar
>>>>  log4j:WARN Error during default initialization
>>>> java.lang.NoClassDefFoundError: org/apache/flume/FlumeException
>>>> at java.lang.Class.getDeclaredConstructors0(Native Method)
>>>>  at java.lang.Class.privateGetDeclaredConstructors(Unknown Source)
>>>> at java.lang.Class.getConstructor0(Unknown Source)
>>>>  at java.lang.Class.newInstance0(Unknown Source)
>>>>
>>>>
>>>> 2.   If I include ng-core, and ng-sdk, I run into the following
>>>> exception.
>>>> Classpath:
>>>> C:\ServiceManager\workspace\mongodb\Mongodb\bin;C:\test\log4j-1.2.16.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-core-1.3.1.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-log4jappender-1.3.1.jar;C:\apache\apache-flume-1.3.1-bin\lib\flume-ng-sdk-1.3.1.jar
>>>>  log4j:WARN Error during default initialization
>>>> java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory
>>>> at
>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168)
>>>>  at
>>>> org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128)
>>>> at
>>>> org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184)