Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Java heap space error while starting flume agent


Copy link to this message
-
Re: Java heap space error while starting flume agent
Wolfgang Hoschek 2014-01-10, 10:44
Flume requires guava.

Wolfgang.

On Jan 10, 2014, at 12:40 PM, Chhaya Vishwakarma wrote:

> Hi,
> My flume version is 1.4.0 and I have not put guava jar in classpath
>
> -----Original Message-----
> From: Wolfgang Hoschek [mailto:[EMAIL PROTECTED]]
> Sent: Friday, January 10, 2014 3:51 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Java heap space error while starting flume agent
>
> Looks like you are running with a guava version that's different than the one that was used to compile. Flume uses guava 11.0.2 per flume/pom.xml.
>
> Wolfgang.
>
> On Jan 10, 2014, at 7:49 AM, Chhaya Vishwakarma wrote:
>
>> Hi
>> Thank you so much that error is gone now I am getting some different error
>>
>>
>> java.lang.VerifyError: class com.google.common.cache.AbstractLoadingCache overrides final method apply.(Ljava/lang/Object;)Ljava/lang/Object;
>>        at java.lang.ClassLoader.defineClass1(Native Method)
>>        at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
>>        at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>>        at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
>>        at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>>        at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
>>        at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>>        at java.security.AccessController.doPrivileged(Native Method)
>>        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>>        at com.cloudera.cdk.morphline.shaded.com.google.common.reflect.ClassPath$ClassInfo.load(ClassPath.java:226)
>>        at com.cloudera.cdk.morphline.api.MorphlineContext.getTopLevelClasses(MorphlineContext.java:127)
>>        at com.cloudera.cdk.morphline.api.MorphlineContext.importCommandBuilders(MorphlineContext.java:68)
>>        at com.cloudera.cdk.morphline.stdlib.Pipe.<init>(Pipe.java:41)
>>        at com.cloudera.cdk.morphline.stdlib.PipeBuilder.build(PipeBuilder.java:39)
>>        at com.cloudera.cdk.morphline.base.Compiler.compile(Compiler.java:125)
>>        at com.cloudera.cdk.morphline.base.Compiler.compile(Compiler.java:54)
>>        at org.apache.flume.sink.solr.morphline.MorphlineHandlerImpl.configure(MorphlineHandlerImpl.java:101)
>>
>>
>> Regards,
>> Chhaya Vishwakarma
>>
>>
>> From: Joao Salcedo [mailto:[EMAIL PROTECTED]]
>> Sent: Friday, January 10, 2014 11:10 AM
>> To: [EMAIL PROTECTED]
>> Subject: Re: Java heap space error while starting flume agent
>>
>> Hi Chhaya,
>>
>> Assuming you are at flume home directory, here is the command to start:
>>
>> bin/flume-ng agent -n agent -f conf/flume-conf.properties -c conf
>>
>> Flume starts and runs with a maximum heap size of 20mb. If you are using batching, you may run into out of memory error pretty quickly. To increase the heap size, rename the file  flume-env.sh.template  to flume-env.sh in conf directory and uncomment the JAVA_OPTS line and edit the minimum and maximum heap. Example: JAVA_OPTS="-Xms512m -Xmx1024m"
>>
>> If "-c conf" is not specified at the end of the command, your flume-env.sh file will not be taken into effect.
>>
>>
>> Cheers,
>>
>> Joao
>>
>> On Fri, Jan 10, 2014 at 4:36 PM, Chhaya Vishwakarma <[EMAIL PROTECTED]> wrote:
>>>
>>> Hi,
>>>
>>>
>>>
>>> I am getting java heap error while starting flume agent how can I increase JVM memory from flume agent?
>>>
>>>
>>>
>>> ERROR java.lang.OutOfMemoryError: Java heap space
>>>
>>>        at java.util.Arrays.copyOf(Arrays.java:2882)
>>>
>>>        at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:100)
>>>
>>>        at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:572)
>>>
>>>        at java.lang.StringBuilder.append(StringBuilder.java:203)