Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - version of avro


Copy link to this message
-
Re: version of avro
Koert Kuipers 2012-10-20, 16:27
option 3 is not available to us. i have been using option 1 without issues
so far (except in hive), but i have only worked with MR1 old api and avro-
generic so far (not sure which of these is relevant...).

On Sat, Oct 20, 2012 at 3:59 AM, Jacob Metcalf <[EMAIL PROTECTED]>wrote:

>
> Yes I have CDH4 working happily with Avro 1.7.0 following the process
> described below.
>
> The various methods of distributing the Avro jar are discussed in
> http://www.cloudera.com/blog/2011/01/how-to-include-third-party-libraries-in-your-map-reduce-job. I
> could not distribute the newer Avro jar via methods 1 and 2 because of
> classpath issues (discussed in AVRO-1103) so settled for the not so
> satisfactory option 3.
>
> The latest version of Avro has a patch for AVRO-1103 and a Maven profile
> to compile against the newer versions of Hadoop. I have not tried upgrading
> yet, but you should definitely try this first before going down the self
> patching route.
>
> Regards
>
> Jacob
>
> ------------------------------
> From: [EMAIL PROTECTED]
> Subject: Re: version of avro
> Date: Fri, 19 Oct 2012 13:16:24 -0700
> To: [EMAIL PROTECTED]
>
>
>
> On Oct 19, 2012, at 1:03pm, Koert Kuipers wrote:
>
> i noticed avro version 1.5.4 is included with some version/distros of
> hadoop and hive... is there a reason why 1.5.4 is included specifically and
> not newer ones? are there some incompatibilities to be aware of? i would
> like to use a newer version
>
>
> In the mail archives there was a discussion back in July about using Avro
> 1.7 with CHD4. where Jacob Metcalf said:
>
> Avro 17 is compiled against Hadoop 1 i.e. CDH3.
>
> I have it running aginst CDH4 but I had to patch and recompile Avro, then
> replace the Avro 153 in the Hadoop lib directory with 17. I have attached
> my patch against JIRA AVRO-1103 for consideration.
>
>
> It sounds like if you're using CDH3/Hadoop 1.0 then you should be able to
> use Avro 1.7 as-is, but I haven't tried this myself.
>
> -- Ken
>
> --------------------------------------------
> http://about.me/kkrugler
> +1 530-210-6378
>
>
>
>
>