Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro RPC in Apache Flume fails with IBM JDK


Copy link to this message
-
Re: Avro RPC in Apache Flume fails with IBM JDK
Hi,
Does anyone have the same problem?
 2013/11/18 1:19 "You Hoken" <[EMAIL PROTECTED]>:

> Hi,
>
> I'm using Avro (1.7.2) RPC in Apache Flume 1.3.1 with the following environment.
>
>  -OS
>    SUSE Linux Enterprise Server 11 (x86_64)
>    VERSION = 11
>    PATCHLEVEL = 2
>
>  -IBM JDK
>   java version "1.6.0"
>   Java(TM) SE Runtime Environment (build pxi3260sr10fp1-20120321_01(SR10
> FP1))
>   IBM J9 VM (build 2.4, JRE 1.6.0 IBM J9 2.4 Linux x86-32
> jvmxi3260sr10fp1-20120202_101568 (JIT enabled, AOT enabled)
>   J9VM - 20120202_101568
>   JIT  - r9_20111107_21307ifx1
>   GC   - 20120202_AA)
>   JCL  - 20120320_01
>
> Avro RPC (AvroSource / AvroSink) is used to collect logs.
>
> My problem is that Avro RPC fails sometimes...
>
> in the case of success in Avro RPC, I could find the following messages in
> "flume.log".
>
> =============================> [TIME] INFO  [lifecycleSupervisor-1-1]
> (org.apache.flume.source.AvroSource.start:143)  - Starting Avro source
> avroSrc: { bindAddress: xxxxx, port: xxxx }...
> [TIME] INFO  [lifecycleSupervisor-1-3]
> (org.apache.flume.source.AvroSource.start:185)  - Avro source avroSrc
> started.
> =============================>
> but in the case of failure, I could not find " Avro source avroSrc started.
> " message.
> # though I found "Starting Avro source avroSrc".
>
> so, I think that the biggest problem is failure of starting Avro source.
>
> and furthermore, implemented debug log in the source code to trace in
> detail, and start and stop repeatedly,
> then I found that MessageDigest#getInstance("MD5") in
> org.apache.avro.Protocol#getMD5() hang up.
>
> does anyone know the cause of this problem and how to solve ?
>
> I guess this problem might be caused when using IBM JDK. how do you think ?
> is this problem related to [AVRO-1221]?
>
> and one more question,
> when Avro source started successfully, I suppose that MD5 digest is same
> value each time.
> so, what does MD5 digest need for ?
>
> regards,
>
> You
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB