Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> filter on a date column?


Copy link to this message
-
Re: filter on a date column?
Hi Tim,

Can't you use ToDate(field, 'format string') instead of casting? For
example, if I have the following record in my input file:

2013/01/01

I can do:

a = LOAD 'input.txt' AS (str:chararray);
b = FOREACH a GENERATE ToDate(str, 'YYYY/MM/DD');
DUMP b;

This gives me:

(2013-01-01T00:00:00.000-08:00)

Thanks,
Cheolsoo

On Thu, Mar 21, 2013 at 9:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:

> Hi Cheolsoo,
>
> What if my date time column is not formatted properly?
> I noticed when I cast my column to datetime, it comes out with null values.
>
> Thanks,
>
> Tim
>
>
> On Thu, Mar 21, 2013 at 4:46 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> wrote:
>
> > Hi Tim,
> >
> > Pig 0.11 has datetime type. You can do something like this:
> >
> > a = LOAD 'input.txt' AS (date:datetime);
> > b = FILTER a BY date < ToDate('2013-01-01');
> >
> > Also see built-in functions for datetime type:
> > http://pig.apache.org/docs/r0.11.0/func.html#datetime-functions
> >
> > Thanks,
> > Cheolsoo
> >
> >
> > On Thu, Mar 21, 2013 at 3:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:
> >
> > > Since there is not date datatype, how do I filter on a date column?
> > > I've been setting the date column as a chararray.
> > >
> > > I would like to do something like:
> > >
> > >
> > > a = filter b by date_col < '2013-01-01';
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB