Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Lzo problem throwing java.io.IOException:java.io.EOFException


Copy link to this message
-
Lzo problem throwing java.io.IOException:java.io.EOFException
I have a bit of an edge case on using lzo which I think might be related
to HIVE-524.
When running a query like this:
select distinct login_cldr_id as cldr_id from chatsessions_load;
I get a java.io.IOException:java.io.EOFException without much of a
description.
I know the output should be a single value and noticed it decided to use
2 reducers.
One of the reducers produced a 0 byte file which I imagine will be the
cause of the IOException. It I do "set mapred.reduce.tasks=1" it works
correctly since there is no 0 byte file anymore.

I also noticed when using gzip I don't see this problem at all.

Since I use