Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> View with map join fails


Copy link to this message
-
Re: View with map join fails
Thank you Bejoy.

I will file a new Jira.

Good wishes,always !
Santosh
On Wed, Jan 9, 2013 at 2:41 AM, <[EMAIL PROTECTED]> wrote:

> **
> Looks like there is a bug with mapjoin + view. Please check hive jira to
> see if there an issue open against this else file a new jira.
>
> From my understanding, When you enable map join, hive parser would create
> back up jobs. These back up jobs are executed only if map join fails. In
> normal cases when map join succeeds these jobs are filtered out and not
> executed.
>
> '1116112419, job is filtered out (removed at runtime).'
>
> Regards
> Bejoy KS
>
> Sent from remote device, Please excuse typos
> ------------------------------
> *From: * Santosh Achhra <[EMAIL PROTECTED]>
> *Date: *Tue, 8 Jan 2013 17:11:18 +0800
> *To: *<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *View with map join fails
>
> Hello,
>
> I have created a view  as shown below.
>
> *CREATE VIEW V1 AS*
> *select /*+ MAPJOIN(t1) ,MAPJOIN(t2)  */ t1.f1, t1.f2, t1.f3, t1.f4,
> t2.f1, t2.f2, t2.f3 from TABLE1 t1 join TABLE t2 on ( t1.f2= t2.f2 and
> t1.f3 = t2.f3 and t1.f4 = t2.f4 ) group by t1.f1, t1.f2, t1.f3, t1.f4,
> t2.f1, t2.f2, t2.f3*
>
> View get created successfully however when I execute below mentioned SQL
> or any SQL on the view  get NULLPOINTER exception error
>
> *hive> select count (*) from V1;*
> *FAILED: NullPointerException null*
> *hive>*
>
> Is there anything wrong with the view creation ?
>
> Next I created view without MAPJOIN hints
>
> *CREATE VIEW V1 AS*
> *select  t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3 from TABLE1 t1
> join TABLE t2 on ( t1.f2= t2.f2 and t1.f3 = t2.f3 and t1.f4 = t2.f4 ) group
> by t1.f1, t1.f2, t1.f3, t1.f4, t2.f1, t2.f2, t2.f3*
>
> Before executing select SQL I excute *set  hive.auto.convert.join=true; *
>
> I am getting beloow mentioned warnings
> java.lang.InstantiationException:
> org.apache.hadoop.hive.ql.parse.ASTNodeOrigin
> Continuing ...
> java.lang.RuntimeException: failed to evaluate: <unbound>=Class.new();
> Continuing ...
>
>
> And I see from log that total 5 mapreduce jobs are
> started however when don't set auto.convert.join to true, I see only 3
> mapreduce jobs getting invoked.
> *Total MapReduce jobs = 5*
> *Ended Job = 1116112419, job is filtered out (removed at runtime).*
> *Ended Job = -33256989, job is filtered out (removed at runtime).*
> *WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated.
> Please use org.apache.hadoop.log.metrics.EventCounter in all the
> log4j.properties files.*
>
>
> Good wishes,always !
> Santosh
>