Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - NullPointerException when inserting into a partition with a select from another table/partition


Copy link to this message
-
NullPointerException when inserting into a partition with a select from another table/partition
Larry Ogrodnek 2009-06-30, 23:43
trying to run the following query:

insert overwrite table imp_test3 partition(ds="20090513") select
d["query_string"] from imp where imp.ds="20090513" and imp.hour="00";

causes all of the tasks in step 2 to fail with:

java.lang.RuntimeException: Map operator initialization failed
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:134)
at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2198)
Caused by: java.lang.NullPointerException
at org.apache.hadoop.hive.ql.exec.MapOperator.initObjectInspector(MapOperator.java:175)

at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:203)
at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:264)
at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:94)

... 3 more
imp_test3 is defined as:
create table imp_test3(qs string)
partitioned by (ds string)
stored as sequencefile
;

If I get rid of the partition in the destination table, and use:
insert overwrite table imp_test3 select d["query_string"] from imp where
imp.ds="20090513" and imp.hour="00";

then everything seems to work....

I also tried adding another partition to the destination table (hour), to
match the input... but, still no go...

imp is defined as:
CREATE TABLE imp (d MAP<STRING,STRING>)
 partitioned by (ds string, hour string)
 ROW FORMAT DELIMITED
 FIELDS TERMINATED BY '\t'
 COLLECTION ITEMS TERMINATED BY '\1'
 MAP KEYS TERMINATED BY '\2'
 STORED AS TEXTFILE
;

thanks for the help.