Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hive add jar question


Copy link to this message
-
Re: hive add jar question
Here is a relevant thread:
http://mail-archives.apache.org/mod_mbox/hive-user/201008.mbox/%3CAANLkTi=+[EMAIL PROTECTED]%3E

I have personally used the "add file" functionality when accessing
resources. You can access them just by their name in your code.

About #2, doesn't sound normal to me. Did you figure that out or still
running into it?

Mark

On Thu, Dec 20, 2012 at 5:01 PM, java8964 java8964 <[EMAIL PROTECTED]> wrote:
> Hi, I have 2 questions related to the hive behavior when using 'add jar'.
>
> I am testing the implementing of my own Hive InputFormat and SerDe in a jar
> in my single machine cluster running in Pseudo distributed mode. In the jar,
> I will include the properties file in the top level of the jar.
>
> In my custom code, I will try to load the properties file through the
> following way:
>
> props.load(this.getClass().getResourceAsStream("my.properties")).
>
> I am sure that the my.properties exists in the my jar file, but
> this.getClass().getResourceAsStream("my.properties") at runtime will return
> NULL in this case. I am not sure the reason for this. Does anyone have an
> idea?
>
> Second question is that when my test data is small, which is less than the
> setting of (hive.exec.mode.local.auto.inputbytes.max, not sure I typed
> correct here), the hive will run my query locally. But in this case, the
> HIVE will fail due to my custom class (like InputFormat) not found error. Of
> course in my session, I did the 'add jar xxxx.jar' command. If the test is
> big, it will run in the standalone cluster without any problem (it finds the
> class in my jar in this case). My question is this normal? Why hive running
> in local mode won't be able to find my class in the jar which is already
> being added?
>
> My environment is CDH3U5, hive will be 0.7.1 in it.
>
>
> Thanks
>
>
> Yong
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB