Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Unixtimestamp


Copy link to this message
-
Re: Unixtimestamp
UnixToISO appears timestamps in milliseconds, not seconds, since the epoch.
The following fragment seems to generate what you're looking for:

file2 = LOAD 'timestamps.txt' USING PigStorage(',') AS (insertDate:long,
changeDate:long,a4);
myF3 = FOREACH file2 GENERATE ISOToYear(UnixToISO(insertDate*1000));
dump myF3;

On an input of:

1154518325,1216638895,0
1139353200,1232213159,0

It generates:

(2006-01-01T00:00:00.000Z)
(2006-01-01T00:00:00.000Z)

Norbert

On Fri, Jul 29, 2011 at 10:01 AM, Marian Condurache <
[EMAIL PROTECTED]> wrote:

> Well i have some logs with unixtimestamps in
> Something like
>
> 1154518325,1216638895,0
> 1139353200,1232213159,0
> ...
> And so on
>
> Then I run:
>
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/piggybank.jar ;
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/lib/joda-time-1.6.jar;
>
> DEFINE UnixToISO
> org.apache.pig.piggybank.evaluation.datetime.convert.UnixToISO();
> DEFINE ISOToYear
> org.apache.pig.piggybank.evaluation.datetime.truncate.ISOToYear();
>
>
> file2 = load '/user/mojo/input/insStatsTest'  USING PigStorage(',')
>                AS (insertDate:long, changeDate:long,a4);
>
>
> myF2 = FOREACH file2  {
>                        x = UnixToISO(insertDate) ;
>                        y =  ISOToYear(x) ;
>                        generate x,y;
>                     }
>
> myF3 = FOREACH file2 generate UnixToISO(insertDate);
>
> rm /user/mcondurache/output/testData
>
> STORE myF3 INTO '/user/mcondurache/output/testData' using PigStorage(',');
>
> And then I get back
>
>
> 1970-01-14T08:41:58.325Z,1970-01-01T00:00:00.000Z
> 1970-01-14T04:29:13.200Z,1970-01-01T00:00:00.000Z
> ...
>
> I am running a built version of pig from your repository and I downloaded
> the joda-time-1.6.jar from sourceforge.net website
>
> ,M
>
> -----Urspr√ľngliche Nachricht-----
> Von: Norbert Burger [mailto:[EMAIL PROTECTED]]
> Gesendet: Freitag, 29. Juli 2011 15:33
> An: [EMAIL PROTECTED]
> Betreff: Re: Unixtimestamp
>
> Seems like the Piggybank's UnixToISO should do what you're looking for - if
> not long, in what datatype is your unixtimestamp arriving to your Pig
> script?
>
> Pig supports explicit casts [1], but I think the conversion you're
> referring
> to should be handled implicitly.
>
> [1] http://ofps.oreilly.com/titles/9781449302641/data_model.html#casts
>
> Norbert
>
> On Fri, Jul 29, 2011 at 9:06 AM, Marian Condurache <
> [EMAIL PROTECTED]> wrote:
>
> > Hi,
> > Any ideas how to convert unixtimestamp to some readable date format like
> > YYYY-MM-DD ?
> > Is there a buit in function ?
> > Because I tried UnixToIso....but it expects unix time long ....or is
> there
> > a way to convert timestamp in time long in Pig ?
> >
> >
> >
> > LG, Marian
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB