Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Unixtimestamp


+
Marian Condurache 2011-07-29, 13:06
+
Norbert Burger 2011-07-29, 13:32
Copy link to this message
-
Re: Unixtimestamp
Norbert Burger 2011-07-29, 14:32
UnixToISO appears timestamps in milliseconds, not seconds, since the epoch.
The following fragment seems to generate what you're looking for:

file2 = LOAD 'timestamps.txt' USING PigStorage(',') AS (insertDate:long,
changeDate:long,a4);
myF3 = FOREACH file2 GENERATE ISOToYear(UnixToISO(insertDate*1000));
dump myF3;

On an input of:

1154518325,1216638895,0
1139353200,1232213159,0

It generates:

(2006-01-01T00:00:00.000Z)
(2006-01-01T00:00:00.000Z)

Norbert

On Fri, Jul 29, 2011 at 10:01 AM, Marian Condurache <
[EMAIL PROTECTED]> wrote:

> Well i have some logs with unixtimestamps in
> Something like
>
> 1154518325,1216638895,0
> 1139353200,1232213159,0
> ...
> And so on
>
> Then I run:
>
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/piggybank.jar ;
> REGISTER /home/mojo/apps/pig/contrib/piggybank/java/lib/joda-time-1.6.jar;
>
> DEFINE UnixToISO
> org.apache.pig.piggybank.evaluation.datetime.convert.UnixToISO();
> DEFINE ISOToYear
> org.apache.pig.piggybank.evaluation.datetime.truncate.ISOToYear();
>
>
> file2 = load '/user/mojo/input/insStatsTest'  USING PigStorage(',')
>                AS (insertDate:long, changeDate:long,a4);
>
>
> myF2 = FOREACH file2  {
>                        x = UnixToISO(insertDate) ;
>                        y =  ISOToYear(x) ;
>                        generate x,y;
>                     }
>
> myF3 = FOREACH file2 generate UnixToISO(insertDate);
>
> rm /user/mcondurache/output/testData
>
> STORE myF3 INTO '/user/mcondurache/output/testData' using PigStorage(',');
>
> And then I get back
>
>
> 1970-01-14T08:41:58.325Z,1970-01-01T00:00:00.000Z
> 1970-01-14T04:29:13.200Z,1970-01-01T00:00:00.000Z
> ...
>
> I am running a built version of pig from your repository and I downloaded
> the joda-time-1.6.jar from sourceforge.net website
>
> ,M
>
> -----Urspr√ľngliche Nachricht-----
> Von: Norbert Burger [mailto:[EMAIL PROTECTED]]
> Gesendet: Freitag, 29. Juli 2011 15:33
> An: [EMAIL PROTECTED]
> Betreff: Re: Unixtimestamp
>
> Seems like the Piggybank's UnixToISO should do what you're looking for - if
> not long, in what datatype is your unixtimestamp arriving to your Pig
> script?
>
> Pig supports explicit casts [1], but I think the conversion you're
> referring
> to should be handled implicitly.
>
> [1] http://ofps.oreilly.com/titles/9781449302641/data_model.html#casts
>
> Norbert
>
> On Fri, Jul 29, 2011 at 9:06 AM, Marian Condurache <
> [EMAIL PROTECTED]> wrote:
>
> > Hi,
> > Any ideas how to convert unixtimestamp to some readable date format like
> > YYYY-MM-DD ?
> > Is there a buit in function ?
> > Because I tried UnixToIso....but it expects unix time long ....or is
> there
> > a way to convert timestamp in time long in Pig ?
> >
> >
> >
> > LG, Marian
> >
>