Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> difference between add jar in hive session and hive --auxpath


Copy link to this message
-
Re: difference between add jar in hive session and hive --auxpath
Essentially anything that is part of the InputFormat needs to be in
auxlib/auxpath. Anything part of a UDF can be added with 'add jar'.

On Fri, Mar 8, 2013 at 1:01 PM, Dean Wampler <
[EMAIL PROTECTED]> wrote:

> --auxpath adds more jars to Hive's classpath before invoking Hive. ADD
> JARS copies jars around the cluster and adds them to the task classpath, so
> the jars you add aren't visible to hive itself. Annoying, but...
>
> On Fri, Mar 8, 2013 at 11:53 AM, java8964 java8964 <[EMAIL PROTECTED]>wrote:
>
>>  This is in HIVE-0.9.0
>>
>> hive> list jars;
>> /nfs_home/common/userlibs/google-collections-1.0.jar
>> /nfs_home/common/userlibs/elephant-bird-hive-3.0.7.jar
>> /nfs_home/common/userlibs/protobuf-java-2.3.0.jar
>> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar
>> file:/usr/lib/hive/lib/hive-builtins-0.9.0-cdh4.1.2.jar
>> hive> desc table;
>> java.lang.NoClassDefFoundError:
>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter
>>         at
>> com.twitter.elephantbird.hive.serde.ProtobufDeserializer.initialize(ProtobufDeserializer.java:45)
>>         at
>> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203)
>>         at
>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260)
>>         at
>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
>>         at
>> org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490)
>>         at
>> org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162)
>>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930)
>>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:844)
>>         at
>> org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2545)
>>         at
>> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:309)
>>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>>         at
>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331)
>>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117)
>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950)
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
>>         at
>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
>>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:744)
>>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: java.lang.ClassNotFoundException:
>> com.twitter.elephantbird.mapreduce.io.ProtobufConverter
>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>         ... 25 more
>> FAILED: Execution Error, return code -101 from
>> org.apache.hadoop.hive.ql.exec.DDLTask
>> hive> exit;
>> [y130zhan@daca2 userlibs]$ jar tvf
>> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar | grep
>> ProtobufConverter
>>   4825 Mon Mar 04 16:50:46 UTC 2013
>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter.class
>>    732 Mon Mar 04 16:50:46 UTC 2013
>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter$1.class
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB