|
|
-
Raise Flag based on join condition
Shin Chan 2012-04-16, 14:12
Hi All,
I have one holiday file and one daily log file.
I have to mark particular day as holiday in daily log file , if that date is matching to holiday File dates
holidayFile = load 'holidayList' as (holiday_date,holiday_description);
dayfile = load 'log' as (log_date);
I want to have output as
dayFileHoliday = log_date , holidayFlag
So sample output could be
12-11-2012,F 12-11-2011,T
When my holiday file has entry for 12-11-2011 as holiday
holiday_date,holiday_description 12-11-2011,School_Closing_day
Since my holidayFile is very small i was thinking to use replcated join.
Any ideas to solve this.
Thanks in advance.
-
Re: Raise Flag based on join condition
Gianmarco De Francisci Mo... 2012-04-16, 14:32
Hi,
just use a replicated outer join, then you can check for the presence of the holiday with a foreach and generate T or F with a ternary conditional operator (i.e. condition ? value_if_true : value_if_false )
Something like:
joined = JOIN dayfile by log_date LEFT, holidayFile by holiday_date USING replicated; filtered = FOREACH joined GENERATE log_date, ( is null holiday_date ? 'F' : 'T' );
Cheers, -- Gianmarco De Francisci Morales
On Mon, Apr 16, 2012 at 16:12, Shin Chan <[EMAIL PROTECTED]> wrote:
> Hi All, > > I have one holiday file and one daily log file. > > I have to mark particular day as holiday in daily log file , if that date > is matching to holiday File dates > > holidayFile = load 'holidayList' as (holiday_date,holiday_description); > > dayfile = load 'log' as (log_date); > > I want to have output as > > dayFileHoliday = log_date , holidayFlag > > So sample output could be > > 12-11-2012,F > 12-11-2011,T > > When my holiday file has entry for 12-11-2011 as holiday > > holiday_date,holiday_description > 12-11-2011,School_Closing_day > > Since my holidayFile is very small i was thinking to use replcated join. > > Any ideas to solve this. > > Thanks in advance. >
-
Re: Raise Flag based on join condition
Shin Chan 2012-04-17, 10:48
Dear Gianmarco De Francisci Morales, Thank You. It worked :) ----- Original Message ----- From: Gianmarco De Francisci Morales Sent: 04/16/12 10:32 AM To: [EMAIL PROTECTED] Subject: Re: Raise Flag based on join condition
Hi, just use a replicated outer join, then you can check for the presence of the holiday with a foreach and generate T or F with a ternary conditional operator (i.e. condition ? value_if_true : value_if_false ) Something like: joined = JOIN dayfile by log_date LEFT, holidayFile by holiday_date USING replicated; filtered = FOREACH joined GENERATE log_date, ( is null holiday_date ? 'F' : 'T' ); Cheers, -- Gianmarco De Francisci Morales On Mon, Apr 16, 2012 at 16:12, Shin Chan <[EMAIL PROTECTED]> wrote: > Hi All, > > I have one holiday file and one daily log file. > > I have to mark particular day as holiday in daily log file , if that date > is matching to holiday File dates > > holidayFile = load 'holidayList' as (holiday_date,holiday_description); > > dayfile = load 'log' as (log_date); > > I want to have output as > > dayFileHoliday = log_date , holidayFlag > > So sample output could be > > 12-11-2012,F > 12-11-2011,T > > When my holiday file has entry for 12-11-2011 as holiday > > ho liday_date,holiday_description > 12-11-2011,School_Closing_day > > Since my holidayFile is very small i was thinking to use replcated join. > > Any ideas to solve this. > > Thanks in advance. >
Thanks and Regards ,
|
|