|
|
Stan Rosenberg 2011-10-12, 23:29
Hi,
I have three constant udfs in jython:
@outputSchema("m:map[bag{tuple()}]") def dummy1(): return {"key":[("value1", "value2")]}
@outputSchema("m:map[tuple()]") def dummy2(): return {"key":("value1", "value2")}
# doesn't work! @outputSchema("m:map[bag{}]") def dummy3(): return {"key":["value1", "value2"]} The first two are fine. The last one which returns a map whose values are lists does not work. (I am getting ClassCastException: java.lang.String cannot be cast to org.apache.pig.data.Tuple). Could somebody explain why?
Thanks,
stan
+
Stan Rosenberg 2011-10-12, 23:29
Jonathan Coveney 2011-10-13, 00:49
Bags have to contain tuples. It's how the type is defined.
2011/10/12 Stan Rosenberg <[EMAIL PROTECTED]>
> Hi, > > I have three constant udfs in jython: > > @outputSchema("m:map[bag{tuple()}]") > def dummy1(): > return {"key":[("value1", "value2")]} > > @outputSchema("m:map[tuple()]") > def dummy2(): > return {"key":("value1", "value2")} > > # doesn't work! > @outputSchema("m:map[bag{}]") > def dummy3(): > return {"key":["value1", "value2"]} > > > The first two are fine. The last one which returns a map whose values > are lists does not work. (I am getting ClassCastException: > java.lang.String cannot be cast to org.apache.pig.data.Tuple). > Could somebody explain why? > > Thanks, > > stan >
+
Jonathan Coveney 2011-10-13, 00:49
Norbert Burger 2011-10-13, 15:52
Also the output schema for dummy3() doesn't match what's being returned. You're returning a list of strings, but the outputschema specifies a bag, which translates into a list of tuples (of something, eg. strings).
Norbert
On Wed, Oct 12, 2011 at 7:29 PM, Stan Rosenberg < [EMAIL PROTECTED]> wrote:
> Hi, > > I have three constant udfs in jython: > > @outputSchema("m:map[bag{tuple()}]") > def dummy1(): > return {"key":[("value1", "value2")]} > > @outputSchema("m:map[tuple()]") > def dummy2(): > return {"key":("value1", "value2")} > > # doesn't work! > @outputSchema("m:map[bag{}]") > def dummy3(): > return {"key":["value1", "value2"]} > > > The first two are fine. The last one which returns a map whose values > are lists does not work. (I am getting ClassCastException: > java.lang.String cannot be cast to org.apache.pig.data.Tuple). > Could somebody explain why? > > Thanks, > > stan >
+
Norbert Burger 2011-10-13, 15:52
Stan Rosenberg 2011-10-13, 19:46
On Thu, Oct 13, 2011 at 11:52 AM, Norbert Burger <[EMAIL PROTECTED]> wrote: > Also the output schema for dummy3() doesn't match what's being returned. > You're returning a list of strings, but the outputschema specifies a bag, > which translates into a list of tuples (of something, eg. strings). >
Part of my question was whether a list of individuals (in python) has a corresponding type in pig. I now see that it doesn't. Instead, it must be converted to a list of tuples.
+
Stan Rosenberg 2011-10-13, 19:46
Thejas Nair 2011-10-13, 20:57
> Part of my question was whether a list of individuals (in python) has > a corresponding type in pig. I now see that it doesn't. Instead, it > must be converted to a list of tuples. There is a patch from Woody to make these translations easier. - https://issues.apache.org/jira/browse/PIG-1942Once there is some consensus on the details, we should have it in. (I hope to spend some time in next few weeks to drive that.) -Thejas
+
Thejas Nair 2011-10-13, 20:57
|
|