Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Weird issue with running tests


Copy link to this message
-
Weird issue with running tests
Hi folks,

I'm running into a weird issue with testing HIVE-3682. I don't think
it's so much to do with the jira at hand itself, as it is to do with
the test or the testing framework.

Basically, if I run the .q file test itself, it succeeds. If I run it
as part of a broader ant test, it fails, and seemingly consistently.
What's more, the reason it fails is that the produced .out file does
not match the golden output in an interesting way. I'm attaching the
two files with this mail if anyone wants to look at it further, but
the diff between them is as follows:

926,929c926
< REHOOK: query: create table array_table (a array<string>, b array<string>)
< ROW FORMAT DELIMITED
< FIELDS TERMINATED BY '\t'
< COLLECTION ITEMS TERMINATED BY ','^@163:val_163^@
---
> 163:val_163
943c940
< REHOOK: type: CREATETABLE^@444:val_444^@
---
> 444:val_444
1027a1025,1029
> PREHOOK: query: create table array_table (a array<string>, b array<string>)
> ROW FORMAT DELIMITED
> FIELDS TERMINATED BY '\t'
> COLLECTION ITEMS TERMINATED BY ','
> PREHOOK: type: CREATETABLE

Note#1 : the PREHOOK log line for a create table seems to have been
logged before the !cat that preceeded it finished logging.
Note#2 : The ^@ separators seem to be indicating a switch between
streams writing out to the .out file.
Note#3 : The "P" in PREHOOK seems to get gobbled up each time this happens.

To me, this looks like !cat runs in a separate thread or at least a
separate PrintStream that it hasn't quite completely flushed its
buffers. Is there a way to force this? I mean, yes, I suppose I can go
edit QTestUtil.execute so as to put an explicit flush, but I won't
know if that works or not till after I do a complete test run(given
that a solo .q run succeeds), and even then, if it succeeds, I won't
know if that is what fixed it.

Has anyone hit something like this before or have any thoughts/theories?

Thanks,
-Sushanth