Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Getting Data into Data Warehouse from Pig


+
rakesh sharma 2012-01-10, 20:42
Copy link to this message
-
Re: Getting Data into Data Warehouse from Pig
Guy Bayes 2012-01-10, 21:27
you might want to take a look at zookeeper as a coordination mechanism for
when to process what file

On Tue, Jan 10, 2012 at 12:42 PM, rakesh sharma <[EMAIL PROTECTED]
> wrote:

>
> Hi All,
> I am quite new to hadoop world and trying to work on a project using
> hadoop and pig. The data is continuously being written in hadoop by many
> producers. All producers concurrently write data to the same file for 30
> minutes duration. After 30 minutes, new file is created and they start
> writing on it. I need to run pig jobs to analyze the data from hadoop
> incrementally and push the resulted data in RDBMS. I am wondering what will
> be the right way to implement it.
> Thanks,RS
+
Dmitriy Ryaboy 2012-01-10, 23:36
+
IGZ Nick 2012-01-11, 06:03
+
IGZ Nick 2012-01-11, 06:15