Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Loading collected files of HDFS sink into Hive


Copy link to this message
-
Loading collected files of HDFS sink into Hive
Hi,

I've got 2 problems while trying to load files generated by HDFS sink into
Hive

1.I set the HDFS sink with hdfs.fileType=DataStream and a table with
TextFile in Hive, when I try to load data from HDFS sink to Hive, I get the
following exception in Hive:

Failed with exception Wrong file format, pease check the file's format

2.How to pick out the closed files and ignore the files with suffix of .tmp
while loading data to Hive?