Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Behaviour on a failed cast.


+
Kris Coward 2013-01-12, 00:04
+
Jameson Li 2013-02-01, 02:45
Copy link to this message
-
Re: Behaviour on a failed cast.

The cast in question was a bytearray to a map (while referencing a value
inside a nested object loaded using JsonLoader), which was failing
because a previous glitch in upstream infrastructure had corrupted some
records. I've long since written a workaround, but if this little bit of
robustness could be made less painful for others to implement in the
future, I'm sure that someone (probably several someones) would
appreciate it.

(And yes, while scouring JIRA for changes that might've tipped me off to
the setting, I did find that the graceful failure to cast was only
implemented for simple data types, so I'd have had to write the
workaround for maps anyway--actually, that's what motivated me to give
up on finding the setting and just write the workaround.)

Cheers,
Kris

On Fri, Feb 01, 2013 at 10:45:48AM +0800, Jameson Li wrote:
> I think If the code is java code in the udf, you'd better catch the
> exception and do something, just not ignore that.
>
> If your pig code meet a cast failed, you can step by step debug your code
> line by line in the interactive mode.
> Sometimes when after one step(as streaming) the schema seems not certain,
> if you describe it and will see that the schema is null.
> And you can using "... as();" to certain the schema and column type, maybe
> you will defeat the cast fail.
>
> 专注于Mysql,MSSQL,Oracle,Hadoop
>
>
>
> 2013/1/12 Kris Coward <[EMAIL PROTECTED]>
>
> > Hi,
> >
> > I remember a while back that there was a setting introduced to allow a
> > pig job to either insert a null, or drop a row, instead of aborting
> > execution, when a cast failed. Of course, now that I need it, I'm having
> > no luck finding it in the list archives, or on google; would someone be
> > so kind as to remind me what the setting is?
> >
> > Thanks,
> > Kris
> >
> > --
> > Kris Coward                                     http://unripe.melon.org/
> > GPG Fingerprint: 2BF3 957D 310A FEEC 4733  830E 21A4 05C7 1FEB 12B3
> >

--
Kris Coward http://unripe.melon.org/
GPG Fingerprint: 2BF3 957D 310A FEEC 4733  830E 21A4 05C7 1FEB 12B3