Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # dev >> Weird issue with running tests


+
Sushanth Sowmyan 2013-04-29, 17:46
+
Ashutosh Chauhan 2013-04-29, 17:55
Copy link to this message
-
Re: Weird issue with running tests
Aha, that makes sense. Thanks!

On Mon, Apr 29, 2013 at 10:55 AM, Ashutosh Chauhan <[EMAIL PROTECTED]> wrote:
> Hi Sushanth,
>
> I would suggest to try dfs -cat in your test instead of !cat, because for !
> we fork a different process, so its possible streams get mangled up, but
> dfs -cat would get you what you want without needing to fork.
>
> Thanks,
> Ashutosh
>
>
> On Mon, Apr 29, 2013 at 10:46 AM, Sushanth Sowmyan <[EMAIL PROTECTED]>wrote:
>
>> Hi folks,
>>
>> I'm running into a weird issue with testing HIVE-3682. I don't think
>> it's so much to do with the jira at hand itself, as it is to do with
>> the test or the testing framework.
>>
>> Basically, if I run the .q file test itself, it succeeds. If I run it
>> as part of a broader ant test, it fails, and seemingly consistently.
>> What's more, the reason it fails is that the produced .out file does
>> not match the golden output in an interesting way. I'm attaching the
>> two files with this mail if anyone wants to look at it further, but
>> the diff between them is as follows:
>>
>> 926,929c926
>> < REHOOK: query: create table array_table (a array<string>, b
>> array<string>)
>> < ROW FORMAT DELIMITED
>> < FIELDS TERMINATED BY '\t'
>> < COLLECTION ITEMS TERMINATED BY ','^@163:val_163^@
>> ---
>> > 163:val_163
>> 943c940
>> < REHOOK: type: CREATETABLE^@444:val_444^@
>> ---
>> > 444:val_444
>> 1027a1025,1029
>> > PREHOOK: query: create table array_table (a array<string>, b
>> array<string>)
>> > ROW FORMAT DELIMITED
>> > FIELDS TERMINATED BY '\t'
>> > COLLECTION ITEMS TERMINATED BY ','
>> > PREHOOK: type: CREATETABLE
>>
>> Note#1 : the PREHOOK log line for a create table seems to have been
>> logged before the !cat that preceeded it finished logging.
>> Note#2 : The ^@ separators seem to be indicating a switch between
>> streams writing out to the .out file.
>> Note#3 : The "P" in PREHOOK seems to get gobbled up each time this happens.
>>
>> To me, this looks like !cat runs in a separate thread or at least a
>> separate PrintStream that it hasn't quite completely flushed its
>> buffers. Is there a way to force this? I mean, yes, I suppose I can go
>> edit QTestUtil.execute so as to put an explicit flush, but I won't
>> know if that works or not till after I do a complete test run(given
>> that a solo .q run succeeds), and even then, if it succeeds, I won't
>> know if that is what fixed it.
>>
>> Has anyone hit something like this before or have any thoughts/theories?
>>
>> Thanks,
>> -Sushanth
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB