Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Hive 0.9.0 column pruner


Copy link to this message
-
Hive 0.9.0 column pruner
By default, the column pruner is set to true:
hive.optimize.cp = true.

However, when we set the hive.optimize.cp = false, then some Hive queries
started to fail.

We get errors like:

java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error
while processing row [Error getting row data with exception
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to
org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryStruct
at org.apache.hadoop.hive.serde2.lazybinary.objectinspector.LazyBinaryStructObjectInspector.getStructFieldData(LazyBinaryStructObjectInspector.java:55)
at org.apache.hadoop.hive.serde2.SerDeUtils.buildJSONString(SerDeUtils.java:349)

 Any insight what is going on here?

P.S. The reason we want to set it to false is that some Hive tables have
data on S3. And seems we have to set column pruner to false to make S3
queries to work. However, by setting it to false, the HDFS queries failed.

Thanks!
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB