Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Set JobClient properties from hive CLI


+
sumit ghosh 2013-04-20, 08:28
Copy link to this message
-
Re: Set JobClient properties from hive CLI
Hi,
 
Sending to a wider audience - How do I set jobclient.progress.monitor.poll.interval in Hive?
 
Thanks,
Sumit

________________________________
From: sumit ghosh <[EMAIL PROTECTED]>
To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Saturday, 20 April 2013 1:28 AM
Subject: Set JobClient properties from hive CLI

Hi,

I am trying to set the property to change the logging interval to more than 5 seconds. Hence I am using the following parameter: jobclient.progress.monitor.poll.interval=5000

It works fine when I use it on an Map Reduce -
$ hadoop jar /usr/lib/hadoop-0.20-mapreduce/hadoop-2.0.0-mr1-cdh4.1.3-examples.jar wordcount -Djobclient.progress.monitor.poll.interval=5000  Input wc
13/04/20 03:23:59 WARN mapred.JobClient: Use GenericOptionsParser for parsing the arguments. Applications should implement Tool for the same.
13/04/20 03:24:00 INFO input.FileInputFormat: Total input paths to process : 1
13/04/20 03:24:00 INFO mapred.JobClient: Running job: job_201304012052_0616
13/04/20 03:24:05 INFO mapred.JobClient:  map 0% reduce 0%
13/04/20 03:24:15 INFO mapred.JobClient:  map 18% reduce 0%
13/04/20 03:24:20 INFO mapred.JobClient:  map 24% reduce 0%
13/04/20 03:24:25 INFO mapred.JobClient:  map 25% reduce 0%
13/04/20 03:24:30 INFO mapred.JobClient:  map 32% reduce 0%
13/04/20 03:24:35 INFO mapred.JobClient:  map 40% reduce 8%
13/04/20 03:24:40 INFO mapred.JobClient:  map 50% reduce 8%
13/04/20 03:24:45 INFO mapred.JobClient:  map 50% reduce 14%
13/04/20 03:24:50 INFO mapred.JobClient:  map 55% reduce 16%

But this property is not working for hive CLI using set or using -hiveconf

hive -e "set jobclient.progress.monitor.poll.interval=5000; select count(*) from Table"
OR
hive -hiveconf jobclient.progress.monitor.poll.interval=5000 -e "select count(*) from Table"

What am I doing wrong?

Thanks,
Sumit.
+
Mike Liddell 2013-04-22, 18:02
+
sumit ghosh 2013-04-22, 22:04
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB