Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - filter on a date column?


Copy link to this message
-
Re: filter on a date column?
Tim Chan 2013-03-22, 06:04
Hi Cheolsoo,

That worked great.

Thanks,

Tim
On Thu, Mar 21, 2013 at 9:41 PM, Cheolsoo Park <[EMAIL PROTECTED]> wrote:

> Hi Tim,
>
> Can't you use ToDate(field, 'format string') instead of casting? For
> example, if I have the following record in my input file:
>
> 2013/01/01
>
> I can do:
>
> a = LOAD 'input.txt' AS (str:chararray);
> b = FOREACH a GENERATE ToDate(str, 'YYYY/MM/DD');
> DUMP b;
>
> This gives me:
>
> (2013-01-01T00:00:00.000-08:00)
>
> Thanks,
> Cheolsoo
>
>
>
> On Thu, Mar 21, 2013 at 9:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:
>
> > Hi Cheolsoo,
> >
> > What if my date time column is not formatted properly?
> > I noticed when I cast my column to datetime, it comes out with null
> values.
> >
> > Thanks,
> >
> > Tim
> >
> >
> > On Thu, Mar 21, 2013 at 4:46 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hi Tim,
> > >
> > > Pig 0.11 has datetime type. You can do something like this:
> > >
> > > a = LOAD 'input.txt' AS (date:datetime);
> > > b = FILTER a BY date < ToDate('2013-01-01');
> > >
> > > Also see built-in functions for datetime type:
> > > http://pig.apache.org/docs/r0.11.0/func.html#datetime-functions
> > >
> > > Thanks,
> > > Cheolsoo
> > >
> > >
> > > On Thu, Mar 21, 2013 at 3:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:
> > >
> > > > Since there is not date datatype, how do I filter on a date column?
> > > > I've been setting the date column as a chararray.
> > > >
> > > > I would like to do something like:
> > > >
> > > >
> > > > a = filter b by date_col < '2013-01-01';
> > > >
> > >
> >
>