Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Lzo problem throwing java.io.IOException:java.io.EOFException


Copy link to this message
-
Lzo problem throwing java.io.IOException:java.io.EOFException
I have a bit of an edge case on using lzo which I think might be related
to HIVE-524.
When running a query like this:
select distinct login_cldr_id as cldr_id from chatsessions_load;
I get a java.io.IOException:java.io.EOFException without much of a
description.
I know the output should be a single value and noticed it decided to use
2 reducers.
One of the reducers produced a 0 byte file which I imagine will be the
cause of the IOException. It I do "set mapred.reduce.tasks=1" it works
correctly since there is no 0 byte file anymore.

I also noticed when using gzip I don't see this problem at all.

Since I use
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB