Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> difference between add jar in hive session and hive --auxpath


Copy link to this message
-
Re: difference between add jar in hive session and hive --auxpath
--auxpath adds more jars to Hive's classpath before invoking Hive. ADD JARS
copies jars around the cluster and adds them to the task classpath, so the
jars you add aren't visible to hive itself. Annoying, but...

On Fri, Mar 8, 2013 at 11:53 AM, java8964 java8964 <[EMAIL PROTECTED]>wrote:

>  This is in HIVE-0.9.0
>
> hive> list jars;
> /nfs_home/common/userlibs/google-collections-1.0.jar
> /nfs_home/common/userlibs/elephant-bird-hive-3.0.7.jar
> /nfs_home/common/userlibs/protobuf-java-2.3.0.jar
> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar
> file:/usr/lib/hive/lib/hive-builtins-0.9.0-cdh4.1.2.jar
> hive> desc table;
> java.lang.NoClassDefFoundError:
> com/twitter/elephantbird/mapreduce/io/ProtobufConverter
>         at
> com.twitter.elephantbird.hive.serde.ProtobufDeserializer.initialize(ProtobufDeserializer.java:45)
>         at
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203)
>         at
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260)
>         at
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
>         at org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490)
>         at
> org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930)
>         at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:844)
>         at
> org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2545)
>         at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:309)
>         at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>         at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331)
>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117)
>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950)
>         at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
>         at
> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
>         at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:744)
>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> Caused by: java.lang.ClassNotFoundException:
> com.twitter.elephantbird.mapreduce.io.ProtobufConverter
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>         ... 25 more
> FAILED: Execution Error, return code -101 from
> org.apache.hadoop.hive.ql.exec.DDLTask
> hive> exit;
> [y130zhan@daca2 userlibs]$ jar tvf
> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar | grep
> ProtobufConverter
>   4825 Mon Mar 04 16:50:46 UTC 2013
> com/twitter/elephantbird/mapreduce/io/ProtobufConverter.class
>    732 Mon Mar 04 16:50:46 UTC 2013
> com/twitter/elephantbird/mapreduce/io/ProtobufConverter$1.class
>
>
> ------------------------------
> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]
> Subject: RE: difference between add jar in hive session and hive --auxpath
> Date: Thu, 7 Mar 2013 16:44:41 -0800
>
>
> If properly done, "add jar <jar-file>" should work the same as passing the
> jar with --auxpath. Can you run "list jars;" command from CLI or Hue and

*Dean Wampler, Ph.D.*
thinkbiganalytics.com
+1-312-339-1330