Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Getting Data into Data Warehouse from Pig

rakesh sharma 2012-01-10, 20:42
Copy link to this message
Re: Getting Data into Data Warehouse from Pig
you might want to take a look at zookeeper as a coordination mechanism for
when to process what file

On Tue, Jan 10, 2012 at 12:42 PM, rakesh sharma <[EMAIL PROTECTED]
> wrote:

> Hi All,
> I am quite new to hadoop world and trying to work on a project using
> hadoop and pig. The data is continuously being written in hadoop by many
> producers. All producers concurrently write data to the same file for 30
> minutes duration. After 30 minutes, new file is created and they start
> writing on it. I need to run pig jobs to analyze the data from hadoop
> incrementally and push the resulted data in RDBMS. I am wondering what will
> be the right way to implement it.
> Thanks,RS
Dmitriy Ryaboy 2012-01-10, 23:36
IGZ Nick 2012-01-11, 06:03
IGZ Nick 2012-01-11, 06:15