Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # dev - RFC: Major HCatalog refactoring


+
Eugene Koifman 2013-08-29, 00:24
Copy link to this message
-
Re: RFC: Major HCatalog refactoring
Eugene Koifman 2013-08-31, 00:41
This will change every file under hcatalog so it has to happen before the
branching.  Most likely at the beginning of next week.

Thanks
On Wed, Aug 28, 2013 at 5:24 PM, Eugene Koifman <[EMAIL PROTECTED]>wrote:

> Hi,
>
>
> Here is the plan for refactoring HCatalog as was agreed to when it was
> merged into Hive during.  HIVE-4869 is the umbrella bug for this work.  The
> changes are complex and touch every single file under hcatalog.  Please
> comment.
>
> When HCatalog project was merged into Hive on 0.11 several integration
> items did not make the 0.11 deadline.  It was agreed to finish them in 0.12
> release.  Specifically:
>
> 1. HIVE-4895 - change package name from org.apache.hcatalog to
> org.apache.hive.hcatalog
>
> 2. HIVE-4896 - create binary backwards compatibility layer for hcat users
> upgrading from 0.11 to 0.12
>
> For item 1, we’ll just move every file under org.apache.hcatalog to
> org.apache.hive.hcatalog and update all “package” and “import” statement as
> well as all hcat/webhcat scripts.  This will include all JUnit tests.
>
> Item 2 will ensure that if a user has a M/R program or Pig script, etc.
> that uses HCatalog public API, their programs will continue to work w/o
> change with hive 0.12.
>
> The proposal is to make the changes that have as little impact on the
> build system, in part to make upcoming ‘mavenization’ of hive easier, in
> part to make the changes more manageable.
>
>
>
> The list of public interfaces (and their transitive closure) for which
> backwards compat will be provided.
>
>    1.
>
>    HCatLoader
>    2.
>
>    HCatStorer
>    3.
>
>    HCatInputFormat
>    4.
>
>    HCatOutputFormat
>    5.
>
>    HCatReader
>    6.
>
>    HCatWriter
>    7.
>
>    HCatRecord
>    8.
>
>    HCatSchema
>
>
> To achieve this, 0.11 version of these classes will be added in
> org.apache.hcatalog package (after item 1 is done).  Each of these classes
> as well as dependencies will be deprecated to make it clear that any new
> development needs to happen in org.apache.hive.hcatalog.  0.11 version of
> JUnit tests for hcat will also be brought to trunk and handled the same way
> as mainline code.  A sunset clause will be added to the deprecation message.
>
> Thus, the published HCatalog JARs will contain both packages and the unit
> tests will cover both versions of the API.
>
> Since these changes are unavoidably disruptive, we’ll need to lock down
> hcatalog part of hive, check in all existing patches (which are ready, i.e.
> apply/test cleanly and don’t have review comments which need to be
> addressed) and them make the refactoring changes.
>
>
> Thanks,
>
> Eugene
>
>
>
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
+
Edward Capriolo 2013-08-31, 14:35
+
Eugene Koifman 2013-08-31, 15:19
+
Edward Capriolo 2013-08-31, 15:59
+
Brock Noland 2013-08-31, 15:30
+
Eugene Koifman 2013-09-02, 15:36
+
Brock Noland 2013-09-03, 14:25
+
Eugene Koifman 2013-09-03, 16:57
+
Edward Capriolo 2013-09-03, 17:22
+
Eugene Koifman 2013-09-03, 18:13
+
Thejas Nair 2013-09-03, 17:20
+
Eugene Koifman 2013-09-03, 19:40