Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> version of avro


Copy link to this message
-
Re: version of avro
option 3 is not available to us. i have been using option 1 without issues
so far (except in hive), but i have only worked with MR1 old api and avro-
generic so far (not sure which of these is relevant...).

On Sat, Oct 20, 2012 at 3:59 AM, Jacob Metcalf <[EMAIL PROTECTED]>wrote:

>
> Yes I have CDH4 working happily with Avro 1.7.0 following the process
> described below.
>
> The various methods of distributing the Avro jar are discussed in
> http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job. I
> could not distribute the newer Avro jar via methods 1 and 2 because of
> classpath issues (discussed in AVRO-1103) so settled for the not so
> satisfactory option 3.
>
> The latest version of Avro has a patch for AVRO-1103 and a Maven profile
> to compile against the newer versions of Hadoop. I have not tried upgrading
> yet, but you should definitely try this first before going down the self
> patching route.
>
> Regards
>
> Jacob
>
> ------------------------------
> From: [EMAIL PROTECTED]
> Subject: Re: version of avro
> Date: Fri, 19 Oct 2012 13:16:24 -0700
> To: [EMAIL PROTECTED]
>
>
>
> On Oct 19, 2012, at 1:03pm, Koert Kuipers wrote:
>
> i noticed avro version 1.5.4 is included with some version/distros of
> hadoop and hive... is there a reason why 1.5.4 is included specifically and
> not newer ones? are there some incompatibilities to be aware of? i would
> like to use a newer version
>
>
> In the mail archives there was a discussion back in July about using Avro
> 1.7 with CHD4. where Jacob Metcalf said:
>
> Avro 17 is compiled against Hadoop 1 i.e. CDH3.
>
> I have it running aginst CDH4 but I had to patch and recompile Avro, then
> replace the Avro 153 in the Hadoop lib directory with 17. I have attached
> my patch against JIRA AVRO-1103 for consideration.
>
>
> It sounds like if you're using CDH3/Hadoop 1.0 then you should be able to
> use Avro 1.7 as-is, but I haven't tried this myself.
>
> -- Ken
>
> --------------------------------------------
> http://about.me/kkrugler
> +1 530-210-6378
>
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB