Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Raise Flag based on join condition


+
Shin Chan 2012-04-16, 14:12
Copy link to this message
-
Re: Raise Flag based on join condition
Hi,

just use a replicated outer join, then you can check for the presence of
the holiday with a foreach and generate T or F with a ternary conditional
operator (i.e. condition ? value_if_true : value_if_false )

Something like:

joined = JOIN dayfile by log_date LEFT, holidayFile by holiday_date USING
replicated;
filtered = FOREACH joined GENERATE log_date, ( is null holiday_date ? 'F' :
'T' );

Cheers,
--
Gianmarco De Francisci Morales

On Mon, Apr 16, 2012 at 16:12, Shin Chan <[EMAIL PROTECTED]> wrote:

> Hi All,
>
>  I have one holiday file and one daily log file.
>
>  I have to mark particular day as holiday in daily log file , if that date
> is matching to holiday File dates
>
>  holidayFile = load 'holidayList' as (holiday_date,holiday_description);
>
>  dayfile = load 'log' as (log_date);
>
>  I want to have output as
>
>  dayFileHoliday = log_date , holidayFlag
>
>  So sample output could be
>
>  12-11-2012,F
>  12-11-2011,T
>
>  When my holiday file has entry for 12-11-2011 as holiday
>
>  holiday_date,holiday_description
>  12-11-2011,School_Closing_day
>
>  Since my holidayFile is very small i was thinking to use replcated join.
>
>  Any ideas to solve this.
>
> Thanks in advance.
>
+
Shin Chan 2012-04-17, 10:48
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB