Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> difference between add jar in hive session and hive --auxpath


Copy link to this message
-
Re: difference between add jar in hive session and hive --auxpath
BTW if your looking for protobuf + hive you should look at...
https://github.com/edwardcapriolo/hive-protobuf/

On Sat, Mar 9, 2013 at 12:32 PM, Ramki Palle <[EMAIL PROTECTED]> wrote:

> I am not sure whether it works but try this:
>
> Put your ADD JAR commands into a file and invoke hive with -i file option.
>
> or
>
> insert your ADD JAR commands in your $HOME/.hiverc file and start hive.
>
>
> -Ramki.
>
>
> On Fri, Mar 8, 2013 at 11:55 PM, Edward Capriolo <[EMAIL PROTECTED]>wrote:
>
>> Essentially anything that is part of the InputFormat needs to be in
>> auxlib/auxpath. Anything part of a UDF can be added with 'add jar'.
>>
>>
>> On Fri, Mar 8, 2013 at 1:01 PM, Dean Wampler <
>> [EMAIL PROTECTED]> wrote:
>>
>>> --auxpath adds more jars to Hive's classpath before invoking Hive. ADD
>>> JARS copies jars around the cluster and adds them to the task classpath, so
>>> the jars you add aren't visible to hive itself. Annoying, but...
>>>
>>> On Fri, Mar 8, 2013 at 11:53 AM, java8964 java8964 <[EMAIL PROTECTED]
>>> > wrote:
>>>
>>>>  This is in HIVE-0.9.0
>>>>
>>>> hive> list jars;
>>>> /nfs_home/common/userlibs/google-collections-1.0.jar
>>>> /nfs_home/common/userlibs/elephant-bird-hive-3.0.7.jar
>>>> /nfs_home/common/userlibs/protobuf-java-2.3.0.jar
>>>> /nfs_home/common/userlibs/elephant-bird-core-3.0.7.jar
>>>> file:/usr/lib/hive/lib/hive-builtins-0.9.0-cdh4.1.2.jar
>>>> hive> desc table;
>>>> java.lang.NoClassDefFoundError:
>>>> com/twitter/elephantbird/mapreduce/io/ProtobufConverter
>>>>         at
>>>> com.twitter.elephantbird.hive.serde.ProtobufDeserializer.initialize(ProtobufDeserializer.java:45)
>>>>         at
>>>> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Table.getDeserializer(Table.java:253)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Table.getCols(Table.java:490)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:162)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930)
>>>>         at
>>>> org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:844)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2545)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:309)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
>>>>         at
>>>> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
>>>>         at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331)
>>>>         at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117)
>>>>         at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950)
>>>>         at
>>>> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
>>>>         at
>>>> org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
>>>>         at
>>>> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
>>>>         at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:744)
>>>>         at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607)
>>>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>>         at
>>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>>>         at
>>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>>>> Caused by: java.lang.ClassNotFoundException:
>>>> com.twitter.elephantbird.mapreduce.io.ProtobufConverter
>>>>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>         at java.security.AccessController.doPrivileged(Native Method)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB