Has anyone tried running HIVE unit tests with Hadoop-1.x. It seems there are some breaking changes in HarFileSystem which is causing unit test failures in Hive. It may not a production issue so are there any recommendations to fix it in hive?

Error:

MetaException(message:Got exception: java.io.IOException URI: har://pfile/test/data/warehouse/srcpart_archived/ds=2008-04-08/hr=12/data.har is an invalid Har URI since '-' not found.  Expecting har://<scheme>-<host>/<path>.)

Thanks,
Kanna

Hi,

I think Carl has a fix committed in HIVE-3025:
https://issues.apache.org/jira/browse/HIVE-3025

Correct me if I am wrong.

Thanks,
Zhenxiao

On Thu, Jun 21, 2012 at 2:31 PM, Kanna Karanam <[EMAIL PROTECTED]> wrote:
> Has anyone tried running HIVE unit tests with Hadoop-1.x. It seems there are some breaking changes in HarFileSystem which is causing unit test failures in Hive. It may not a production issue so are there any recommendations to fix it in hive?
>
> Error:
>
> MetaException(message:Got exception: java.io.IOException URI: har://pfile/test/data/warehouse/srcpart_archived/ds=2008-04-08/hr=12/data.har is an invalid Har URI since '-' not found.  Expecting har://<scheme>-<host>/<path>.)
>
> Thanks,
> Kanna

All indications are most of the things work fine, but we seem to have
a steady trickle of compatibility issues slightly after things are
released. I can bet your issue is unescaped - in uris

On Thu, Jun 21, 2012 at 5:36 PM, Zhenxiao Luo <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I think Carl has a fix committed in HIVE-3025:
> https://issues.apache.org/jira/browse/HIVE-3025
>
> Correct me if I am wrong.
>
> Thanks,
> Zhenxiao
>
> On Thu, Jun 21, 2012 at 2:31 PM, Kanna Karanam <[EMAIL PROTECTED]> wrote:
>> Has anyone tried running HIVE unit tests with Hadoop-1.x. It seems there are some breaking changes in HarFileSystem which is causing unit test failures in Hive. It may not a production issue so are there any recommendations to fix it in hive?
>>
>> Error:
>>
>> MetaException(message:Got exception: java.io.IOException URI: har://pfile/test/data/warehouse/srcpart_archived/ds=2008-04-08/hr=12/data.har is an invalid Har URI since '-' not found.  Expecting har://<scheme>-<host>/<path>.)
>>
>> Thanks,
>> Kanna

Thanks Zhenxiao - Yes you are right. This patch should fix my issue. I don't see these changes yet in trunk. So it would be a great help if we can get these changes ASAP as it is blocking my windows porting work.

Thanks,
Kanna

-----Original Message-----
From: Edward Capriolo [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 21, 2012 2:39 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Test failures : Running HIVE unit test with Hadoop - 1.x

All indications are most of the things work fine, but we seem to have a steady trickle of compatibility issues slightly after things are released. I can bet your issue is unescaped - in uris

On Thu, Jun 21, 2012 at 5:36 PM, Zhenxiao Luo <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I think Carl has a fix committed in HIVE-3025:
> https://issues.apache.org/jira/browse/HIVE-3025
>
> Correct me if I am wrong.
>
> Thanks,
> Zhenxiao
>
> On Thu, Jun 21, 2012 at 2:31 PM, Kanna Karanam <[EMAIL PROTECTED]> wrote:
>> Has anyone tried running HIVE unit tests with Hadoop-1.x. It seems there are some breaking changes in HarFileSystem which is causing unit test failures in Hive. It may not a production issue so are there any recommendations to fix it in hive?
>>
>> Error:
>>
>> MetaException(message:Got exception: java.io.IOException URI:
>> har://pfile/test/data/warehouse/srcpart_archived/ds=2008-04-08/hr=12/
>> data.har is an invalid Har URI since '-' not found.  Expecting
>> har://<scheme>-<host>/<path>.)
>>
>> Thanks,
>> Kanna

Kanna (image see below)

http://www.edwardcapriolo.com/roller/edwardcapriolo/resource/hive_issues.png

We have had a recent spree of new issues from supporting new versions
of hadoop, and your efforts to support windows.

I have some of your tickets on my todo list and will tackle the
commits as soon as possible, im on the case.

Edward
On Thu, Jun 21, 2012 at 6:27 PM, Kanna Karanam <[EMAIL PROTECTED]> wrote:
> Thanks Zhenxiao - Yes you are right. This patch should fix my issue. I don't see these changes yet in trunk. So it would be a great help if we can get these changes ASAP as it is blocking my windows porting work.
>
> Thanks,
> Kanna
>
> -----Original Message-----
> From: Edward Capriolo [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, June 21, 2012 2:39 PM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Test failures : Running HIVE unit test with Hadoop - 1.x
>
> All indications are most of the things work fine, but we seem to have a steady trickle of compatibility issues slightly after things are released. I can bet your issue is unescaped - in uris
>
> On Thu, Jun 21, 2012 at 5:36 PM, Zhenxiao Luo <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> I think Carl has a fix committed in HIVE-3025:
>> https://issues.apache.org/jira/browse/HIVE-3025
>>
>> Correct me if I am wrong.
>>
>> Thanks,
>> Zhenxiao
>>
>> On Thu, Jun 21, 2012 at 2:31 PM, Kanna Karanam <[EMAIL PROTECTED]> wrote:
>>> Has anyone tried running HIVE unit tests with Hadoop-1.x. It seems there are some breaking changes in HarFileSystem which is causing unit test failures in Hive. It may not a production issue so are there any recommendations to fix it in hive?
>>>
>>> Error:
>>>
>>> MetaException(message:Got exception: java.io.IOException URI:
>>> har://pfile/test/data/warehouse/srcpart_archived/ds=2008-04-08/hr=12/
>>> data.har is an invalid Har URI since '-' not found.  Expecting
>>> har://<scheme>-<host>/<path>.)
>>>
>>> Thanks,
>>> Kanna
>
>

Thanks Edward. I will take this patch and apply it internally on my devbox to unblock myself. No hurry. :)

Zhenxiao - It seems some of these q.out files contains paths related to UNIX so we may need to mask them before committing this patch to trunk. (it may be a problem on Windows)

Thanks,
Kanna

-----Original Message-----
From: Edward Capriolo [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 21, 2012 3:39 PM
To: [EMAIL PROTECTED]
Cc: [EMAIL PROTECTED]
Subject: Re: Test failures : Running HIVE unit test with Hadoop - 1.x

Kanna (image see below)

http://www.edwardcapriolo.com/roller/edwardcapriolo/resource/hive_issues.png

We have had a recent spree of new issues from supporting new versions of hadoop, and your efforts to support windows.

I have some of your tickets on my todo list and will tackle the commits as soon as possible, im on the case.

Edward
On Thu, Jun 21, 2012 at 6:27 PM, Kanna Karanam <[EMAIL PROTECTED]> wrote:
> Thanks Zhenxiao - Yes you are right. This patch should fix my issue. I don't see these changes yet in trunk. So it would be a great help if we can get these changes ASAP as it is blocking my windows porting work.
>
> Thanks,
> Kanna
>
> -----Original Message-----
> From: Edward Capriolo [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, June 21, 2012 2:39 PM
> To: [EMAIL PROTECTED]
> Cc: [EMAIL PROTECTED]
> Subject: Re: Test failures : Running HIVE unit test with Hadoop - 1.x
>
> All indications are most of the things work fine, but we seem to have
> a steady trickle of compatibility issues slightly after things are
> released. I can bet your issue is unescaped - in uris
>
> On Thu, Jun 21, 2012 at 5:36 PM, Zhenxiao Luo <[EMAIL PROTECTED]> wrote:
>> Hi,
>>
>> I think Carl has a fix committed in HIVE-3025:
>> https://issues.apache.org/jira/browse/HIVE-3025
>>
>> Correct me if I am wrong.
>>
>> Thanks,
>> Zhenxiao
>>
>> On Thu, Jun 21, 2012 at 2:31 PM, Kanna Karanam <[EMAIL PROTECTED]> wrote:
>>> Has anyone tried running HIVE unit tests with Hadoop-1.x. It seems there are some breaking changes in HarFileSystem which is causing unit test failures in Hive. It may not a production issue so are there any recommendations to fix it in hive?
>>>
>>> Error:
>>>
>>> MetaException(message:Got exception: java.io.IOException URI:
>>> har://pfile/test/data/warehouse/srcpart_archived/ds=2008-04-08/hr=12
>>> / data.har is an invalid Har URI since '-' not found.  Expecting
>>> har://<scheme>-<host>/<path>.)
>>>
>>> Thanks,
>>> Kanna
>
>