Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Merging different HDFS file for HIVE


Copy link to this message
-
Merging different HDFS file for HIVE
Hi,

Please help in providing solution for the below problem... this scenario is
applicable in Banking atleast...

I have a HIVE table with the below structure...

Hive Table:
Field1
...
Field 10
For the above table, I will get the values for each feed in different file.
You can imagine that these files belongs to same branch and will get at any
time interval. I have to load into table only if I get all 3 files for the
same branch. (assume that we have a common field in all the files to join)

*Feed file 1 :*
EMP ID
Field 1
Field 2
Field 6
Field 9

*Feed File2 :*
EMP ID
Field 5
Field 7
Field 10

*Feed File3 :*
EMP ID
Field 3
Field 4
Field 8

Now the question is,
what is the best way to make all these files to make it as a single file so
that it can be placed under the HIVE structure.

regards,
Rams