Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # dev >> Weird issue with running tests


+
Sushanth Sowmyan 2013-04-29, 17:46
+
Ashutosh Chauhan 2013-04-29, 17:55
Copy link to this message
-
Re: Weird issue with running tests
Aha, that makes sense. Thanks!

On Mon, Apr 29, 2013 at 10:55 AM, Ashutosh Chauhan <[EMAIL PROTECTED]> wrote:
> Hi Sushanth,
>
> I would suggest to try dfs -cat in your test instead of !cat, because for !
> we fork a different process, so its possible streams get mangled up, but
> dfs -cat would get you what you want without needing to fork.
>
> Thanks,
> Ashutosh
>
>
> On Mon, Apr 29, 2013 at 10:46 AM, Sushanth Sowmyan <[EMAIL PROTECTED]>wrote:
>
>> Hi folks,
>>
>> I'm running into a weird issue with testing HIVE-3682. I don't think
>> it's so much to do with the jira at hand itself, as it is to do with
>> the test or the testing framework.
>>
>> Basically, if I run the .q file test itself, it succeeds. If I run it
>> as part of a broader ant test, it fails, and seemingly consistently.
>> What's more, the reason it fails is that the produced .out file does
>> not match the golden output in an interesting way. I'm attaching the
>> two files with this mail if anyone wants to look at it further, but
>> the diff between them is as follows:
>>
>> 926,929c926
>> < REHOOK: query: create table array_table (a array<string>, b
>> array<string>)
>> < ROW FORMAT DELIMITED
>> < FIELDS TERMINATED BY '\t'
>> < COLLECTION ITEMS TERMINATED BY ','^@163:val_163^@
>> ---
>> > 163:val_163
>> 943c940
>> < REHOOK: type: CREATETABLE^@444:val_444^@
>> ---
>> > 444:val_444
>> 1027a1025,1029
>> > PREHOOK: query: create table array_table (a array<string>, b
>> array<string>)
>> > ROW FORMAT DELIMITED
>> > FIELDS TERMINATED BY '\t'
>> > COLLECTION ITEMS TERMINATED BY ','
>> > PREHOOK: type: CREATETABLE
>>
>> Note#1 : the PREHOOK log line for a create table seems to have been
>> logged before the !cat that preceeded it finished logging.
>> Note#2 : The ^@ separators seem to be indicating a switch between
>> streams writing out to the .out file.
>> Note#3 : The "P" in PREHOOK seems to get gobbled up each time this happens.
>>
>> To me, this looks like !cat runs in a separate thread or at least a
>> separate PrintStream that it hasn't quite completely flushed its
>> buffers. Is there a way to force this? I mean, yes, I suppose I can go
>> edit QTestUtil.execute so as to put an explicit flush, but I won't
>> know if that works or not till after I do a complete test run(given
>> that a solo .q run succeeds), and even then, if it succeeds, I won't
>> know if that is what fixed it.
>>
>> Has anyone hit something like this before or have any thoughts/theories?
>>
>> Thanks,
>> -Sushanth
>>