Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Merging different HDFS file for HIVE


Copy link to this message
-
Merging different HDFS file for HIVE
Ramasubramanian Narayanan... 2013-07-26, 10:52
Hi,

Please help in providing solution for the below problem... this scenario is
applicable in Banking atleast...

I have a HIVE table with the below structure...

Hive Table:
Field1
...
Field 10
For the above table, I will get the values for each feed in different file.
You can imagine that these files belongs to same branch and will get at any
time interval. I have to load into table only if I get all 3 files for the
same branch. (assume that we have a common field in all the files to join)

*Feed file 1 :*
EMP ID
Field 1
Field 2
Field 6
Field 9

*Feed File2 :*
EMP ID
Field 5
Field 7
Field 10

*Feed File3 :*
EMP ID
Field 3
Field 4
Field 8

Now the question is,
what is the best way to make all these files to make it as a single file so
that it can be placed under the HIVE structure.

regards,
Rams
+
Nitin Pawar 2013-07-26, 12:30
+
Stephen Sprague 2013-07-26, 23:37
+
Sanjay Subramanian 2013-07-27, 01:23
+
Sanjay Subramanian 2013-07-27, 01:30