Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # dev >> Review Request: SQOOP-428: Support compression for Avro import


Copy link to this message
-
Re: Review Request: SQOOP-428: Support compression for Avro import


> On 2012-01-28 00:13:35, Tom White wrote:
> > src/docs/user/import.txt, line 390
> > <https://reviews.apache.org/r/3600/diff/2/?file=70839#file70839line390>
> >
> >     Should we also support the full classname as an alias so we can avoid this note?
> >    
> >     See also https://issues.apache.org/jira/browse/HADOOP-7323 which will support short names in Hadoop too (from 23 I believe).

Good idea, shouldn't be that hard I hope. I'll look into implementing someting like "getShortName" in Sqoop's CodecMap.

I'll try to submit a new patch by Monday.
- Lars
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3600/#review4666
-----------------------------------------------------------
On 2012-01-26 15:43:36, Lars Francke wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3600/
> -----------------------------------------------------------
>
> (Updated 2012-01-26 15:43:36)
>
>
> Review request for Sqoop.
>
>
> Summary
> -------
>
> This basically only ports all the code from Avro's (1.5.4) AvroOutputFormat to the new MR API.
>
> I've changed the test to extract the common functionality into a helper method because they are the same apart from the two command line arguments.
>
> I could have deleted AvroJob completely but as I was told last time that binary compatibility needs to be maintained I left it in. It's not needed anymore as all necessary functionality can be gotten from Avro's own version of that file as far as I can tell. So if it's okay to delete that redundant file (two actually, cloudera and apache package) let me know and I'll provide a new patch.
>
>
> This addresses bug SQOOP-428.
>     https://issues.apache.org/jira/browse/SQOOP-428
>
>
> Diffs
> -----
>
>   src/docs/user/import.txt deddb1a
>   src/java/org/apache/sqoop/mapreduce/AvroJob.java a57aaf1
>   src/java/org/apache/sqoop/mapreduce/AvroOutputFormat.java 96befd7
>   src/java/org/apache/sqoop/mapreduce/ImportJobBase.java ed6954a
>   src/test/com/cloudera/sqoop/TestAvroImport.java 1b8b046
>
> Diff: https://reviews.apache.org/r/3600/diff
>
>
> Testing
> -------
>
> All tests pass for hadoopversion=20 but TestColumnTypes fails for me on 23. I can't see how that's related though.
>
>
> Thanks,
>
> Lars
>
>