Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Hive, datanucleus, jdbc, localmode.


Copy link to this message
-
Re: Hive, datanucleus, jdbc, localmode.
This article describes exactly what hive test does.

:)
On Sat, Dec 28, 2013 at 9:18 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:

> Anyone try this yet :
>
>
> http://hadoop-pig-hive-thejas.blogspot.com/2013/04/running-hive-in-local-mode.html
>
> ?
>
>
> On Sat, Dec 28, 2013 at 8:09 PM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>
>> -Local mode should have clear instructions on how to run fully local hive
>> jobs, with no hadoop installation.
>>
>> -I like the hive_test repo but I'm not yet sure hive_test is 100% up to
>> date with the simplest strategy for testing hive workflows on the JVM.
>>
>>
>> On Dec 28, 2013, at 4:19 PM, Lefty Leverenz <[EMAIL PROTECTED]>
>> wrote:
>>
>> This sounds like something the documentation should cover.
>>
>>    - What information should be added to the Local Mode<https://cwiki.apache.org/confluence/display/Hive/AdminManual+MetastoreAdmin#AdminManualMetastoreAdmin-LocalMetastore>section?
>>    - Should the wiki have a link to hive_test<https://github.com/edwardcapriolo/hive_test> (for
>>    example, in Hive Developer FAQ<https://cwiki.apache.org/confluence/display/Hive/HiveDeveloperFAQ>
>>    )?
>>
>>
>> -- Lefty
>>
>>
>> On Sat, Dec 28, 2013 at 8:02 AM, Edward Capriolo <[EMAIL PROTECTED]>wrote:
>>
>>> I do not think so. Local mode, just implies the job tracker is local
>>> (and some of the temp storage directories) it does not imply hive will use
>>> hadoop without forking.
>>>
>>>
>>> On Sat, Dec 28, 2013 at 10:43 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
>>>
>>>> Thanks... But are you sure this is the only way? Or is there some magic
>>>> way to run hive in local mode that we both are missing out on ?:)...
>>>>
>>>> - isn't hive in local mode supposed to be run simply via the
>>>> jdbc://hive URL which runs local mode... Or maybe by the fork config
>>>> parameter?
>>>>
>>>> - For example see the parameters in this file:
>>>>
>>>>
>>>> https://github.com/riptano/brisk/blob/master/resources/hive/conf/hive-site.xml
>>>>
>>>>
>>>> On Dec 28, 2013, at 10:22 AM, Edward Capriolo <[EMAIL PROTECTED]>
>>>> wrote:
>>>>
>>>> You can follow along to what I do here.
>>>>
>>>> https://github.com/edwardcapriolo/hive_test
>>>>
>>>> Essentially hive requires a HADOOP_HOME because it always wants to fork
>>>> a bin/hadoop process. Hive-test helps you unpack hadoop inside target and
>>>> change your hadoop_home to some other directory.
>>>>
>>>> It would be nice if there was some other way to do this.
>>>>
>>>>
>>>> On Fri, Dec 27, 2013 at 10:27 PM, Jay Vyas <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Hi Hive:
>>>>>
>>>>> I'm attempting to create a robust eclipse based dev environment for
>>>>> testing my hive jobs in localmode however I run into classnotfound errors
>>>>> depending on which maven dependencies I use. Also, it seems when I change
>>>>> these dependencies from hive 0.12 to hive 0.11, I get other errors related
>>>>> to hive trying to launch jobs via calling /usr/bin/hadoop.
>>>>>
>>>>> This I am stuck: I can't run hive 12 in local java mode because of
>>>>> subtle datanucleus class and API inconsistencies which are tough to
>>>>> resolve, and when going to hive 11, it seems local mode is not natively
>>>>> detected via the jdbc URL...
>>>>>
>>>>> So I have 2 questions:
>>>>>
>>>>> 0) how does hive 12 versus 11 implement local mode differently ?
>>>>>
>>>>> And
>>>>>
>>>>> 1) What is the right way to in hive in pure java/ local environments?
>>>>>
>>>>> The hive book suggests modifying configuration properties, for local
>>>>> mode..
>>>>>
>>>>> but I also have found  that in hive 0.12 , using the jdbc://hive
>>>>> connection URL automagically launches jobs in local mode..
>>>>>
>>>>> However in 0.11 , I see calls to /usr/bin/hadoop when running java
>>>>> classes in local eclipse environment.
>>>>>
>>>>> Thanks!
>>>>>
>>>>> FYI to see an example of my pom.xml, you can checkout the
>>>>> github://jayunit100/bigpetstore pom.xml file.
>>>>>
>>>>
>>>>
>>>
>>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com