Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> version of avro


+
Koert Kuipers 2012-10-19, 20:03
+
Ken Krugler 2012-10-19, 20:16
Copy link to this message
-
RE: version of avro


Yes I have CDH4 working happily with Avro 1.7.0 following the process described below.
The various methods of distributing the Avro jar are discussed in http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job. I could not distribute the newer Avro jar via methods 1 and 2 because of classpath issues (discussed in AVRO-1103) so settled for the not so satisfactory option 3.
The latest version of Avro has a patch for AVRO-1103 and a Maven profile to compile against the newer versions of Hadoop. I have not tried upgrading yet, but you should definitely try this first before going down the self patching route.
Regards
Jacob

From: [EMAIL PROTECTED]
Subject: Re: version of avro
Date: Fri, 19 Oct 2012 13:16:24 -0700
To: [EMAIL PROTECTED]
On Oct 19, 2012, at 1:03pm, Koert Kuipers wrote:i noticed avro version 1.5.4 is included with some version/distros of hadoop and hive... is there a reason why 1.5.4 is included specifically and not newer ones? are there some incompatibilities to be aware of? i would like to use a newer version

In the mail archives there was a discussion back in July about using Avro 1.7 with CHD4. where Jacob Metcalf said:
Avro 17 is compiled against Hadoop 1 i.e. CDH3.

I have it running aginst CDH4 but I had to patch and recompile Avro, then replace the Avro 153 in the Hadoop lib directory with 17. I have attached my patch against JIRA AVRO-1103 for consideration.

It sounds like if you're using CDH3/Hadoop 1.0 then you should be able to use Avro 1.7 as-is, but I haven't tried this myself.
-- Ken

--------------------------------------------http://about.me/kkrugler+1 530-210-6378
     
+
Koert Kuipers 2012-10-20, 16:27
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB