Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Raise Flag based on join condition


Copy link to this message
-
Re: Raise Flag based on join condition
Hi,

just use a replicated outer join, then you can check for the presence of
the holiday with a foreach and generate T or F with a ternary conditional
operator (i.e. condition ? value_if_true : value_if_false )

Something like:

joined = JOIN dayfile by log_date LEFT, holidayFile by holiday_date USING
replicated;
filtered = FOREACH joined GENERATE log_date, ( is null holiday_date ? 'F' :
'T' );

Cheers,
--
Gianmarco De Francisci Morales

On Mon, Apr 16, 2012 at 16:12, Shin Chan <[EMAIL PROTECTED]> wrote:

> Hi All,
>
>  I have one holiday file and one daily log file.
>
>  I have to mark particular day as holiday in daily log file , if that date
> is matching to holiday File dates
>
>  holidayFile = load 'holidayList' as (holiday_date,holiday_description);
>
>  dayfile = load 'log' as (log_date);
>
>  I want to have output as
>
>  dayFileHoliday = log_date , holidayFlag
>
>  So sample output could be
>
>  12-11-2012,F
>  12-11-2011,T
>
>  When my holiday file has entry for 12-11-2011 as holiday
>
>  holiday_date,holiday_description
>  12-11-2011,School_Closing_day
>
>  Since my holidayFile is very small i was thinking to use replcated join.
>
>  Any ideas to solve this.
>
> Thanks in advance.
>