Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> HIVE: java.lang.ArrayIndexOutOfBoundsException: 2 during JOIN


+
Fernando Andrés Doglio Tu... 2013-01-15, 18:31
+
Mark Grover 2013-01-15, 18:35
Copy link to this message
-
Re: HIVE: java.lang.ArrayIndexOutOfBoundsException: 2 during JOIN
The "data_table" has around 5k fields, all doubles.
As for the "age_mean" table, here it is:

hive> desc age_mean;
OK
id string
name string
age_mean double
Time taken: 0.127 seconds

Does this help?

Thanks!
Fernando

On Tue, Jan 15, 2013 at 4:35 PM, Mark Grover <[EMAIL PROTECTED]>wrote:

> Fernando,
> Could you share your table definitions as well please?
>
>
> On Tue, Jan 15, 2013 at 10:31 AM, Fernando Andrés Doglio Turissini <
> [EMAIL PROTECTED]> wrote:
>
>> Hello everyone, I'm struggling with an exception I'm getting on a
>> particular query that's driving me crazy!
>>
>> Here is the exception I get:
>>
>> java.lang.RuntimeException:
>> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
>> processing writable org.apache.hadoop.hive.serde2.colum
>> nar.BytesRefArrayWritable@71412b61
>>         at
>> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
>>         at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>>         at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:441)
>>         at org.apache.hadoop.mapred.MapTask.run(MapTask.java:377)
>>         at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>>         at java.security.AccessController.doPrivileged(Native Method)
>>         at javax.security.auth.Subject.doAs(Subject.java:396)
>>         at
>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1132)
>>         at org.apache.hadoop.mapred.Child.main(Child.java:249)
>> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
>> Error while processing writable
>> org.apache.hadoop.hive.serde2.columnar.BytesRefArray
>> Writable@71412b61
>>         at
>> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:524)
>>         at
>> org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
>>         ... 8 more
>> Caused by: java.lang.ArrayIndexOutOfBoundsException: 2
>>         at
>> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:506)
>>         ... 9 more
>>
>>
>> Here is the query I'm running:
>>
>> INSERT INTO TABLE variance
>> SELECT id, collect_set(name)[0], SUM( POW(age - age_mean, 2) ) / count(1)
>> FROM age_mean join data_table on (age_mean.id = '01' AND data_table.q1 >> 1)
>> where age is not null and age_mean is not null GROUP BY id;
>>
>> It's probably relevant to mention that I'm doing this on an EMR cluster.
>>
>> Any idea what might be causing the exception?
>>
>> Thanks!
>> Fernando
>>
>
>
+
Mark Grover 2013-01-15, 18:46
+
Fernando Andrés Doglio Tu... 2013-01-15, 18:59
+
Fernando Andrés Doglio Tu... 2013-01-15, 20:34
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB