Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Set JobClient properties from hive CLI


+
sumit ghosh 2013-04-20, 08:28
+
sumit ghosh 2013-04-22, 01:39
+
Mike Liddell 2013-04-22, 18:02
Copy link to this message
-
Re: Set JobClient properties from hive CLI
Thanks Mike.

________________________________
From: Mike Liddell <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; sumit ghosh <[EMAIL PROTECTED]>
Sent: Monday, 22 April 2013 11:02 AM
Subject: RE: Set JobClient properties from hive CLI

Dropping [EMAIL PROTECTED]
 
try “set hive.exec.counters.pull.interval = 5000;”
Ref: ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java::progress()
 
-mike.
From:sumit ghosh [mailto:[EMAIL PROTECTED]]
Sent: Sunday, April 21, 2013 6:40 PM
To: [EMAIL PROTECTED]; sumit ghosh; [EMAIL PROTECTED]
Subject: Re: Set JobClient properties from hive CLI
 
Hi,
 
Sending to a wider audience - How do I set jobclient.progress.monitor.poll.interval in Hive?
 
Thanks,
Sumit
From:sumit ghosh <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Saturday, 20 April 2013 1:28 AM
Subject: Set JobClient properties from hive CLI
 
Hi,
 
I am trying to set the property to change the logging interval to more than 5 seconds. Hence I am using the following parameter: jobclient.progress.monitor.poll.interval=5000
 
It works fine when I use it on an Map Reduce -
$ hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.1.3-examples.jar wordcount -Djobclient.progress.monitor.poll.interval=5000  Input wc
13/04/20 03:23:59 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/04/20 03:24:00 INFO input.FileInputFormat: Total input paths to process : 1
13/04/20 03:24:00 INFO mapred.JobClient: Running job: job_201304012052_0616
13/04/20 03:24:05 INFO mapred.JobClient:  map 0% reduce 0%
13/04/20 03:24:15 INFO mapred.JobClient:  map 18% reduce 0%
13/04/20 03:24:20 INFO mapred.JobClient:  map 24% reduce 0%
13/04/20 03:24:25 INFO mapred.JobClient:  map 25% reduce 0%
13/04/20 03:24:30 INFO mapred.JobClient:  map 32% reduce 0%
13/04/20 03:24:35 INFO mapred.JobClient:  map 40% reduce 8%
13/04/20 03:24:40 INFO mapred.JobClient:  map 50% reduce 8%
13/04/20 03:24:45 INFO mapred.JobClient:  map 50% reduce 14%
13/04/20 03:24:50 INFO mapred.JobClient:  map 55% reduce 16%
But this property is not working for hive CLI using set or using -hiveconf
 
hive -e "set jobclient.progress.monitor.poll.interval=5000; select count(*) from Table"
OR
hive -hiveconf jobclient.progress.monitor.poll.interval=5000 -e "select count(*) from Table"
 
What am I doing wrong?
 
Thanks,
Sumit.