Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> filter on a date column?


Copy link to this message
-
Re: filter on a date column?
Hi Cheolsoo,

That worked great.

Thanks,

Tim
On Thu, Mar 21, 2013 at 9:41 PM, Cheolsoo Park <[EMAIL PROTECTED]> wrote:

> Hi Tim,
>
> Can't you use ToDate(field, 'format string') instead of casting? For
> example, if I have the following record in my input file:
>
> 2013/01/01
>
> I can do:
>
> a = LOAD 'input.txt' AS (str:chararray);
> b = FOREACH a GENERATE ToDate(str, 'YYYY/MM/DD');
> DUMP b;
>
> This gives me:
>
> (2013-01-01T00:00:00.000-08:00)
>
> Thanks,
> Cheolsoo
>
>
>
> On Thu, Mar 21, 2013 at 9:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:
>
> > Hi Cheolsoo,
> >
> > What if my date time column is not formatted properly?
> > I noticed when I cast my column to datetime, it comes out with null
> values.
> >
> > Thanks,
> >
> > Tim
> >
> >
> > On Thu, Mar 21, 2013 at 4:46 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hi Tim,
> > >
> > > Pig 0.11 has datetime type. You can do something like this:
> > >
> > > a = LOAD 'input.txt' AS (date:datetime);
> > > b = FILTER a BY date < ToDate('2013-01-01');
> > >
> > > Also see built-in functions for datetime type:
> > > http://pig.apache.org/docs/r0.11.0/func.html#datetime-functions
> > >
> > > Thanks,
> > > Cheolsoo
> > >
> > >
> > > On Thu, Mar 21, 2013 at 3:11 PM, Tim Chan <[EMAIL PROTECTED]> wrote:
> > >
> > > > Since there is not date datatype, how do I filter on a date column?
> > > > I've been setting the date column as a chararray.
> > > >
> > > > I would like to do something like:
> > > >
> > > >
> > > > a = filter b by date_col < '2013-01-01';
> > > >
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB