Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Why does Generic Options Parser only take the first -D option?


Copy link to this message
-
Re: Why does Generic Options Parser only take the first -D option?
I checked GenericOptionsParser from 0.20.2
processGeneralOptions() should be able to process all -D options:

    if (line.hasOption('D')) {
*      String[] property = line.getOptionValues('D');
*      for(String prop : property) {
        String[] keyval = prop.split("=", 2);
        if (keyval.length == 2) {
          conf.set(keyval[0], keyval[1]);
        }
      }
    }
You can add a log after the bold line to verify that all -D options are
returned.

On Thu, Sep 2, 2010 at 10:09 AM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> This is 0.20.0
> I have an eclipse run configuration passing these as arguments
> -D hive2rdbms.jdbc.driver="com.mysql.jdbc.Driver" -D
> hive2rdbms.connection.url="jdbc:mysql://localhost:3306/test" -D
> hive2rdbms.data.query="SELECT id,name FROM name WHERE $CONDITIONS" -D
> hive2rdbms.bounding.query="SELECT min(id),max(id) FROM name" -D
> hive2rdbms.output.strategy=HDFS -D hive2rdbms.ouput.hdfs.path="/tmp/a"
>
> My code does this:
>        public int run(String[] args) throws Exception {
>
>                conf = getConf();
>                GenericOptionsParser parser = new
> GenericOptionsParser(conf,args);
>
>                for (String arg: parser.getRemainingArgs()){
>                  System.out.println(arg);
>                }
>
> hive2rdbms.connection.url=jdbc:mysql://localhost:3306/test
> -D
> hive2rdbms.data.query=SELECT id,name FROM name WHERE $CONDITIONS
> -D
> hive2rdbms.bounding.query=SELECT min(id),max(id) FROM name
> -D
> hive2rdbms.output.strategy=HDFS
> -D
> hive2rdbms.ouput.hdfs.path=/tmp/a
> 10/09/02 13:04:04 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId> Exception in thread "main" java.io.IOException:
> hive2rdbms.connection.url not specified
>        at
> com.media6.hive2rdbms.job.Rdbms2Hive.checkArgs(Rdbms2Hive.java:70)
>        at com.media6.hive2rdbms.job.Rdbms2Hive.run(Rdbms2Hive.java:46)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>        at com.media6.hive2rdbms.job.Rdbms2Hive.main(Rdbms2Hive.java:145)
>
> So what gives does GenericOptionsParser only take hadoop arguments
> like mapred.map.tasks? If so how come it sucks up my first -D argument
> and considers the other ones "Remaining Arguments".
>
> Any ideas?
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB