Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> AW: Get the job id for a hive query


+
Tim Bittersohl 2013-02-28, 14:56
+
Tim Bittersohl 2013-02-28, 11:39
+
Tim Bittersohl 2013-02-28, 11:48
Copy link to this message
-
AW: Get the job id for a hive query
Ok, it works.

For testing, I fired a create table command and a select without a where
clause. Both don’t result in MapReduce jobs... with a where clause, there is
a job created now.

 

Thanks

 

 

Von: Nitin Pawar [mailto:[EMAIL PROTECTED]]
Gesendet: Dienstag, 5. März 2013 12:48
An: [EMAIL PROTECTED]
Betreff: Re: Get the job id for a hive query

 

if the job is submitted to hadoop, it will come up on jobtracker.

Unless you are slow on tracking and your job history retaining is 0, each
hive query submitted to jobtracker will be there on jobhistory

 

On Tue, Mar 5, 2013 at 4:58 PM, Tim Bittersohl <[EMAIL PROTECTED]> wrote:

Hi,

 

I do have the following problem monitoring my Hive queries in Hadoop.

 

I create a Sever using the Hive library which connects to an Hadoop cluster
(file system, job tracker and Hive metastore are set up on this cluster).
The needed parameters for the Hive server I've set in the configuration.

Commands sent to this server are executed, that works.

The problem now is that the MapReduce jobs created by Hive don't appear in
the job tracker's job list. The list itself works, showing me other commands
I executed not using Hive. (I use the JobClient Java class to get the list)

 

Why are the Hive query jobs not in the list? How can I track them in Hadoop?

 

 

Thanks

 

 

Von: Nitin Pawar [mailto:[EMAIL PROTECTED]]
Gesendet: Donnerstag, 28. Februar 2013 16:13
An: [EMAIL PROTECTED]
Betreff: Re: Get the job id for a hive query

 

you can set this property mapred.job.name and this should set the name for
the job

 

On Thu, Feb 28, 2013 at 8:26 PM, Tim Bittersohl <[EMAIL PROTECTED]> wrote:

Thanks for the response,

 

I also found no way to access the job id via java thrift client, all I can
get is a query ID by the query planner.

 

How to set the name of a job where a Hive query is fired with, so I can find
it in the job tracker later?

 

Tim Bittersohl

Software Engineer
http://www.innoplexia.de/ci/logo/inno_logo_links%20200x80.png

Innoplexia GmbH
Mannheimer Str. 175

69123 Heidelberg

Tel.: +49 (0) 6221 7198033 <tel:%2B49%20%280%29%206221%207198033>  
Fax: +49 (0) 6221 7198034 <tel:%2B49%20%280%29%206221%207198034>  
Web: www.innoplexia.com <http://www.innoplexia.com/>  

Sitz: 69123 Heidelberg, Mannheimer Str. 175 - Steuernummer 32494/62606 -
USt. IdNr.: DE 272 871 728 - Geschäftsführer: Christian Schneider, Walery
Strauch, Prof. Dr. Herbert Schuster

 

Von: Nitin Pawar [mailto:[EMAIL PROTECTED]]
Gesendet: Donnerstag, 28. Februar 2013 15:46
An: [EMAIL PROTECTED]
Betreff: Re: Get the job id for a hive query

 

With thrift Client I don't think you can get the jobID from hadoop (I may
very well be wrong in this)

 

the other way around this was to have a separate name for each job you fire
through hive and then directly query jobtracker for the same

 

On Thu, Feb 28, 2013 at 5:18 PM, Tim Bittersohl <[EMAIL PROTECTED]> wrote:

I use java and there HiveClient of the hive library (Version 0.10.0).

 

Tim Bittersohl

Software Engineer
http://www.innoplexia.de/ci/logo/inno_logo_links%20200x80.png

Innoplexia GmbH
Mannheimer Str. 175

69123 Heidelberg

Tel.: +49 (0) 6221 7198033 <tel:%2B49%20%280%29%206221%207198033>  
Fax: +49 (0) 6221 7198034 <tel:%2B49%20%280%29%206221%207198034>  
Web: www.innoplexia.com <http://www.innoplexia.com/>  

Sitz: 69123 Heidelberg, Mannheimer Str. 175 - Steuernummer 32494/62606 -
USt. IdNr.: DE 272 871 728 - Geschäftsführer: Christian Schneider, Walery
Strauch, Prof. Dr. Herbert Schuster

 

Von: Nitin Pawar [mailto:[EMAIL PROTECTED]]
Gesendet: Donnerstag, 28. Februar 2013 12:43
An: [EMAIL PROTECTED]
Betreff: Re: Get the job id for a hive query

 

how are you running your hive queries? using hive cli or hive jdbc client?

 

if you are using hive cli then whenever you fire a query and assuming it is
syntactically correct and its not select * from table operation

hive cli shows  the job ID and an URL which points to the job tracker url
for the job it submitted  

 

On Thu, Feb 28, 2013 at 5:09 PM, Tim Bittersohl <[EMAIL PROTECTED]> wrote:

I’m trying to get the job id of the job created with a Hive query.

At the moment I can get the cluster status from the HiveClient, but I don’
find any job id in there...

 

Tim Bittersohl

Software Engineer
http://www.innoplexia.de/ci/logo/inno_logo_links%20200x80.png

Innoplexia GmbH
Mannheimer Str. 175

69123 Heidelberg

Tel.: +49 (0) 6221 7198033 <tel:%2B49%20%280%29%206221%207198033>  
Fax: +49 (0) 6221 7198034 <tel:%2B49%20%280%29%206221%207198034>  
Web: www.innoplexia.com <http://www.innoplexia.com/>  

Sitz: 69123 Heidelberg, Mannheimer Str. 175 - Steuernummer 32494/62606 -
USt. IdNr.: DE 272 871 728 - Geschäftsführer: Christian Schneider, Walery
Strauch, Prof. Dr. Herbert Schuster

 

Von: Harsh J [mailto:[EMAIL PROTECTED]]
Gesendet: Donnerstag, 28. Februar 2013 09:08
An: hive request
Betreff: Re: Get the job id for a hive query

 

The client logs print the Job ID of the spawned job and a tracking URL. Is
that what you're looking for? Its printed for each stage.

 

On Wed, Feb 27, 2013 at 11:06 PM, Tim Bittersohl <[EMAIL PROTECTED]> wrote:

Hi,

 

has the Hive client the possibility to give back the job id of the job
created when running a query? I need that for tracking.

 

 

Greetings

Tim Bittersohl

Software Engineer
http://www.innoplexia.de/ci/logo/inno_logo_links%20200x80.png

Innoplexia GmbH
Mannheimer Str. 175

69123 Heidelberg

Tel.: +49 (0) 6221 7198033 <tel:%2B49%20%280%29%206221%207198033>  
Fax: +49 (0) 6221 7198034 <tel:%2B49%20%280%29%206221%207198034>  
Web: www.innoplexia.com <http://www.innoplexia.com/>  

Sitz: 69123 Heidelberg, Mannheimer Str. 175 - Steuernummer 32494/62606
<tel:32494%2F62606>  - USt. IdNr.: DE 272 871 728 - Geschäftsführer:
Christian Schneider, Walery Strauch, Prof. Dr. Herbert Schuster

 

 

Harsh J

 

Nitin Pawar

 

Niti
+
Tim Bittersohl 2013-03-05, 11:28