Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Creating Indexes


+
Peter Marron 2012-10-31, 17:42
+
Shreepadma Venugopalan 2012-10-31, 18:37
+
Peter Marron 2012-10-31, 19:52
+
Shreepadma Venugopalan 2012-10-31, 21:58
+
Peter Marron 2012-11-01, 09:31
+
Dean Wampler 2012-11-01, 13:01
+
Bejoy KS 2012-11-01, 13:09
+
Dean Wampler 2012-11-01, 13:20
+
Peter Marron 2012-11-02, 08:44
Copy link to this message
-
Re: Creating Indexes
Dean Wampler 2012-11-02, 14:02
Oh, I saw this line in your Hive output and just assumed you were running
in a cluster:

Hadoop job information for Stage-1: number of mappers: 511; number of
reducers: 138****

I haven't tried running a job that big in pseudodistributed mode either,
but that's beside the point.

So it seems to be an issue with indexing, but it still begs the question
why derby isn't on the classpath for the task. I would try using the ADD
JAR command, which copies the jar around the "cluster" and puts it on the
classpath. It's what you would use with UDFs, for example:

ADD JAR /path/to/derby.jar
ALTER INDEX ...;

It's a huge hack, but it just might work.
dean

On Fri, Nov 2, 2012 at 3:44 AM, Peter Marron <
[EMAIL PROTECTED]> wrote:

>  Hi Dean,****
>
> ** **
>
> I’m running everything on a single physical machine in pseudo-distributed
> mode.****
>
> ** **
>
> Well it certainly looks like the reducer is looking for a derby.jar,
> although I must****
>
> confess I don’t really understand why it would be doing that.****
>
> In an effort to fix that I copied the derby.jar (derby-10.4.2.0.jar) into
> the****
>
> Hadoop directory, where I assume that the reducer would be able to find it.
> ****
>
> However I get exactly the same problem as before.****
>
> Is there some particular place that I should put the derby.jar to make this
> ****
>
> problem go away? Is there anything else that I can try?****
>
> ** **
>
> Peter Marron****
>
> ** **
>
> *From:* Dean Wampler [mailto:[EMAIL PROTECTED]]
> *Sent:* 01 November 2012 13:02
>
> *To:* [EMAIL PROTECTED]
> *Subject:* Re: Creating Indexes****
>
> ** **
>
> It looks like you're using Derby with a real cluster, not just a single
> machine in local or pseudo-distributed mode. I haven't tried this myself,
> but the derby jar is probably not on the machine that ran the reducer task
> that failed.****
>
> ** **
>
> dean****
>
> On Thu, Nov 1, 2012 at 4:31 AM, Peter Marron <
> [EMAIL PROTECTED]> wrote:****
>
> Hi Shreepadma,****
>
>  ****
>
> I agree that the error looks odd. However I can’t believe that I would have
> ****
>
> got this far with Hive if there was no derby jar. Nevertheless I checked.*
> ***
>
> Here is a directory listing of the Hive install:****
>
>  [snip]****
>
> ** **
>

--
*Dean Wampler, Ph.D.*
thinkbiganalytics.com
+1-312-339-1330
+
Peter Marron 2012-11-02, 23:59
+
Shreepadma Venugopalan 2012-11-03, 00:06
+
Peter Marron 2012-11-07, 13:33
+
Dean Wampler 2012-11-03, 00:02
+
Dean Wampler 2012-11-03, 00:03
+
Peter Marron 2012-11-23, 08:56