Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Set JobClient properties from hive CLI


Copy link to this message
-
Re: Set JobClient properties from hive CLI
Thanks Mike.

________________________________
From: Mike Liddell <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; sumit ghosh <[EMAIL PROTECTED]>
Sent: Monday, 22 April 2013 11:02 AM
Subject: RE: Set JobClient properties from hive CLI

Dropping [EMAIL PROTECTED]
 
try “set hive.exec.counters.pull.interval = 5000;”
Ref: ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java::progress()
 
-mike.
From:sumit ghosh [mailto:[EMAIL PROTECTED]]
Sent: Sunday, April 21, 2013 6:40 PM
To: [EMAIL PROTECTED]; sumit ghosh; [EMAIL PROTECTED]
Subject: Re: Set JobClient properties from hive CLI
 
Hi,
 
Sending to a wider audience - How do I set jobclient.progress.monitor.poll.interval in Hive?
 
Thanks,
Sumit
From:sumit ghosh <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Saturday, 20 April 2013 1:28 AM
Subject: Set JobClient properties from hive CLI
 
Hi,
 
I am trying to set the property to change the logging interval to more than 5 seconds. Hence I am using the following parameter: jobclient.progress.monitor.poll.interval=5000
 
It works fine when I use it on an Map Reduce -
$ hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.1.3-examples.jar wordcount -Djobclient.progress.monitor.poll.interval=5000  Input wc
13/04/20 03:23:59 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/04/20 03:24:00 INFO input.FileInputFormat: Total input paths to process : 1
13/04/20 03:24:00 INFO mapred.JobClient: Running job: job_201304012052_0616
13/04/20 03:24:05 INFO mapred.JobClient:  map 0% reduce 0%
13/04/20 03:24:15 INFO mapred.JobClient:  map 18% reduce 0%
13/04/20 03:24:20 INFO mapred.JobClient:  map 24% reduce 0%
13/04/20 03:24:25 INFO mapred.JobClient:  map 25% reduce 0%
13/04/20 03:24:30 INFO mapred.JobClient:  map 32% reduce 0%
13/04/20 03:24:35 INFO mapred.JobClient:  map 40% reduce 8%
13/04/20 03:24:40 INFO mapred.JobClient:  map 50% reduce 8%
13/04/20 03:24:45 INFO mapred.JobClient:  map 50% reduce 14%
13/04/20 03:24:50 INFO mapred.JobClient:  map 55% reduce 16%
But this property is not working for hive CLI using set or using -hiveconf
 
hive -e "set jobclient.progress.monitor.poll.interval=5000; select count(*) from Table"
OR
hive -hiveconf jobclient.progress.monitor.poll.interval=5000 -e "select count(*) from Table"
 
What am I doing wrong?
 
Thanks,
Sumit.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB