Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> filter on a date column?


Copy link to this message
-
Re: filter on a date column?
Hi Tim,

Can't you use ToDate(field, 'format string') instead of casting? For
example, if I have the following record in my input file:

2013/01/01

I can do:

a = LOAD 'input.txt' AS (str:chararray);
b = FOREACH a GENERATE ToDate(str, 'YYYY/MM/DD');
DUMP b;

This gives me:

(2013-01-01T00:00:00.000-08:00)

Thanks,
Cheolsoo

On Thu, Mar 21, 2013 at 9:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:

> Hi Cheolsoo,
>
> What if my date time column is not formatted properly?
> I noticed when I cast my column to datetime, it comes out with null values.
>
> Thanks,
>
> Tim
>
>
> On Thu, Mar 21, 2013 at 4:46 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> wrote:
>
> > Hi Tim,
> >
> > Pig 0.11 has datetime type. You can do something like this:
> >
> > a = LOAD 'input.txt' AS (date:datetime);
> > b = FILTER a BY date < ToDate('2013-01-01');
> >
> > Also see built-in functions for datetime type:
> > http://pig.apache.org/docs/r0.11.0/func.html#datetime-functions
> >
> > Thanks,
> > Cheolsoo
> >
> >
> > On Thu, Mar 21, 2013 at 3:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:
> >
> > > Since there is not date datatype, how do I filter on a date column?
> > > I've been setting the date column as a chararray.
> > >
> > > I would like to do something like:
> > >
> > >
> > > a = filter b by date_col < '2013-01-01';
> > >
> >
>