Hi,
Our company current is using CDH3 release, which comes with Hive 0.7.1.
Right now, I have the data coming from another team, which also provides the custom InputFormat and RecorderReader, but using the new mapreduce API.
I am trying to build a hive table on these data, and hope I can reuse the existing InputFormat. But I got a problem, that in HIVE 0.7.x, only old API is supported.
I have 3 options now:
1) Upgrade whole hadoop system,which is not a real option as right now.2) Ask the other team to provide the custom InputFormat/RecorderReader under old mapred API. I am working on this, but not sure if it is possible or how long.3) Writable a wrapper class, providing old API based on the new API class. My question is, is the 3rd optional valid? Does anyone do the similar things before?

Thanks
Yong    

Yong,
In my opinion, #2 wouldn't be that hard to do.

Of course, upgrading to the new API (albeit by updating Hive) would the
right thing to do long term.

Mark

On Mon, Dec 3, 2012 at 7:42 AM, java8964 java8964 <[EMAIL PROTECTED]>wrote:

>  Hi,
>
> Our company current is using CDH3 release, which comes with Hive 0.7.1.
>
> Right now, I have the data coming from another team, which also provides
> the custom InputFormat and RecorderReader, but using the new mapreduce API.
>
> I am trying to build a hive table on these data, and hope I can reuse the
> existing InputFormat. But I got a problem, that in HIVE 0.7.x, only old API
> is supported.
>
> I have 3 options now:
>
> 1) Upgrade whole hadoop system,which is not a real option as right now.
> 2) Ask the other team to provide the custom InputFormat/RecorderReader
> under old mapred API. I am working on this, but not sure if it is possible
> or how long.
> 3) Writable a wrapper class, providing old API based on the new API class.
> My question is, is the 3rd optional valid? Does anyone do
> the similar things before?
>
>
> Thanks
>
> Yong
>

this is an unfortunate issue with hive we only support mapred api not
mapreduce. If you look at the hbase and cassandra support you see that
sonetimes you can wrap a mapreduce input format by extending the new
interface and implementing the old one. So it is possible to use a newer
api but its ugly code wise. Cleaner would be to produce two distinct input
formats.

On Tuesday, December 4, 2012, Mark Grover <[EMAIL PROTECTED]>
wrote:
> Yong,
> In my opinion, #2 wouldn't be that hard to do.
> Of course, upgrading to the new API (albeit by updating Hive) would the
right thing to do long term.
> Mark
> On Mon, Dec 3, 2012 at 7:42 AM, java8964 java8964 <[EMAIL PROTECTED]>
wrote:
>>
>> Hi,
>> Our company current is using CDH3 release, which comes with Hive 0.7.1.
>> Right now, I have the data coming from another team, which also provides
the custom InputFormat and RecorderReader, but using the new mapreduce API.
>> I am trying to build a hive table on these data, and hope I can reuse
the existing InputFormat. But I got a problem, that in HIVE 0.7.x, only old
API is supported.
>> I have 3 options now:
>> 1) Upgrade whole hadoop system,which is not a real option as right now.
>> 2) Ask the other team to provide the custom InputFormat/RecorderReader
under old mapred API. I am working on this, but not sure if it is possible
or how long.
>> 3) Writable a wrapper class, providing old API based on the new API
class. My question is, is the 3rd optional valid? Does anyone do
the similar things before?
>>
>> Thanks
>> Yong
>