Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - difference between add jar in hive session and hive --auxpath


Copy link to this message
-
Re: difference between add jar in hive session and hive --auxpath
Ramki Palle 2013-03-09, 17:32
I am not sure whether it works but try this:

Put your ADD JAR commands into a file and invoke hive with -i file option.

or

insert your ADD JAR commands in your $HOME/.hiverc file and start hive.
-Ramki.
On Fri, Mar 8, 2013 at 11:55 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:

> Essentially anything that is part of the InputFormat needs to be in
> auxlib/auxpath. Anything part of a UDF can be added with 'add jar'.
>
>
> On Fri, Mar 8, 2013 at 1:01 PM, Dean Wampler <
> [EMAIL PROTECTED]> wrote:
>
>> --auxpath adds more jars to Hive's classpath before invoking Hive. ADD
>> JARS copies jars around the cluster and adds them to the task classpath, so
>> the jars you add aren't visible to hive itself. Annoying, but...
>>
>> On Fri, Mar 8, 2013 at 11:53 AM, java8964 java8964 <[EMAIL PROTECTED]>wrote:
>>
>>>  This is in HIVE-0.9.0
>>>
>>> hive> list jars;
>>> /nfs_home/common/userlibs/google-collections-1.0.jar
>>> /nfs_home/common/userlibs/elephant-bird-hive-3.0.7.jar
>>> /nfs_home/common/userlibs/protobuf-java-2.3.0.jar
>>> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar
>>> file:/usr/lib/hive/lib/hive-builtins-0.9.0-cdh4.1.2.jar
>>> hive> desc table;
>>> java.lang.NoClassDefFoundError:
>>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter
>>>         at
>>> com.twitter.elephantbird.hive.serde.ProtobufDeserializer.initialize(ProtobufDeserializer.java:45)
>>>         at
>>> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203)
>>>         at
>>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260)
>>>         at
>>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
>>>         at
>>> org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490)
>>>         at
>>> org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162)
>>>         at
>>> org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930)
>>>         at
>>> org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:844)
>>>         at
>>> org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2545)
>>>         at
>>> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:309)
>>>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>>>         at
>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331)
>>>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117)
>>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950)
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
>>>         at
>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
>>>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:744)
>>>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607)
>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>         at
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>         at
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>> Caused by: java.lang.ClassNotFoundException:
>>> com.twitter.elephantbird.mapreduce.io.ProtobufConverter
>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>         at java.security.AccessController.doPrivileged(Native Method)
>>>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>         ... 25 more
>>> FAILED: Execution Error, return code -101 from
>>> org.apache.hadoop.hive.ql.exec.DDLTask