Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Pulling off server's debug info to Client


Copy link to this message
-
Re: Pulling off server's debug info to Client
Navis류승우 2012-07-06, 00:19
I've modified hive to notify jobId for monitoring purpose. Check the issue
and vote for acceptance ^^

https://issues.apache.org/jira/browse/HIVE-3235

2012/7/6 VanHuy Pham <[EMAIL PROTECTED]>

> Hi all,
>     I have been interested in getting the debug information and display it
> to clients.
>     Specifically, when you submit a job to hive cli (on terminal say you
> write "select * from somewhere ...) or through a thrift server, you will
> see they hive displays some debug information on the screen (See below)
>
>     How I can I get this "live" information to the client while the
> job is running so the client can see the stages of the job and how it
> is going, see if the job gets killed, or fails... I am using thrift
> server, and see this information on the thrift server's terminal and
> wonder if I can transfer this info to thrift client. Or if it's
> impossible to transfer this info to thrift client, how would one
> direct this debug info somewhere to local file or mysql then the
> information can be read from there?
>
> Any ideas?
> Thanks
>
>
>
> ------------------------------------------------------
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks is set to 0 since there's no reduce operator
> ---> getPaths = XXXXXXXXXXXX
> Starting Job = XXXXXXXXXXXXXX, Tracking URL = XXXXXXXXXXXXXXXXXX
> Kill Command = XXXXXXXXXXXXXXXXXXXX
> Hadoop job information for Stage-1: number of mappers: 505; number of
> reducers: 0
> 2012-07-04 01:52:42,145 Stage-1 map = 0%,  reduce = 0%
> 2012-07-04 01:52:48,185 Stage-1 map = 5%,  reduce = 0%, Cumulative CPU
> 42.92 sec
> 2012-07-04 01:52:49,196 Stage-1 map = 26%,  reduce = 0%, Cumulative
> CPU 213.03 sec
>