Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # dev - Weird issue with running tests


+
Sushanth Sowmyan 2013-04-29, 17:46
Copy link to this message
-
Re: Weird issue with running tests
Ashutosh Chauhan 2013-04-29, 17:55
Hi Sushanth,

I would suggest to try dfs -cat in your test instead of !cat, because for !
we fork a different process, so its possible streams get mangled up, but
dfs -cat would get you what you want without needing to fork.

Thanks,
Ashutosh
On Mon, Apr 29, 2013 at 10:46 AM, Sushanth Sowmyan <[EMAIL PROTECTED]>wrote:

> Hi folks,
>
> I'm running into a weird issue with testing HIVE-3682. I don't think
> it's so much to do with the jira at hand itself, as it is to do with
> the test or the testing framework.
>
> Basically, if I run the .q file test itself, it succeeds. If I run it
> as part of a broader ant test, it fails, and seemingly consistently.
> What's more, the reason it fails is that the produced .out file does
> not match the golden output in an interesting way. I'm attaching the
> two files with this mail if anyone wants to look at it further, but
> the diff between them is as follows:
>
> 926,929c926
> < REHOOK: query: create table array_table (a array<string>, b
> array<string>)
> < ROW FORMAT DELIMITED
> < FIELDS TERMINATED BY '\t'
> < COLLECTION ITEMS TERMINATED BY ','^@163:val_163^@
> ---
> > 163:val_163
> 943c940
> < REHOOK: type: CREATETABLE^@444:val_444^@
> ---
> > 444:val_444
> 1027a1025,1029
> > PREHOOK: query: create table array_table (a array<string>, b
> array<string>)
> > ROW FORMAT DELIMITED
> > FIELDS TERMINATED BY '\t'
> > COLLECTION ITEMS TERMINATED BY ','
> > PREHOOK: type: CREATETABLE
>
> Note#1 : the PREHOOK log line for a create table seems to have been
> logged before the !cat that preceeded it finished logging.
> Note#2 : The ^@ separators seem to be indicating a switch between
> streams writing out to the .out file.
> Note#3 : The "P" in PREHOOK seems to get gobbled up each time this happens.
>
> To me, this looks like !cat runs in a separate thread or at least a
> separate PrintStream that it hasn't quite completely flushed its
> buffers. Is there a way to force this? I mean, yes, I suppose I can go
> edit QTestUtil.execute so as to put an explicit flush, but I won't
> know if that works or not till after I do a complete test run(given
> that a solo .q run succeeds), and even then, if it succeeds, I won't
> know if that is what fixed it.
>
> Has anyone hit something like this before or have any thoughts/theories?
>
> Thanks,
> -Sushanth
>
+
Sushanth Sowmyan 2013-04-29, 17:56