Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Merging different HDFS file for HIVE


Copy link to this message
-
Merging different HDFS file for HIVE
Hi,

Please help in providing solution for the below problem... this scenario is
applicable in Banking atleast...

I have a HIVE table with the below structure...

Hive Table:
Field1
...
Field 10
For the above table, I will get the values for each feed in different file.
You can imagine that these files belongs to same branch and will get at any
time interval. I have to load into table only if I get all 3 files for the
same branch. (assume that we have a common field in all the files to join)

*Feed file 1 :*
EMP ID
Field 1
Field 2
Field 6
Field 9

*Feed File2 :*
EMP ID
Field 5
Field 7
Field 10

*Feed File3 :*
EMP ID
Field 3
Field 4
Field 8

Now the question is,
what is the best way to make all these files to make it as a single file so
that it can be placed under the HIVE structure.

regards,
Rams
+
Nitin Pawar 2013-07-26, 12:30
+
Stephen Sprague 2013-07-26, 23:37
+
Sanjay Subramanian 2013-07-27, 01:23
+
Sanjay Subramanian 2013-07-27, 01:30
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB