|
|
Hi everybody,
I was trying to run a simple script:
A = LOAD 'test01' AS (f1:chararray,f2:int,f3:chararray);
B = order A by f1,f2,f3 DESC;
but, suddenly I got this exception
2012-10-06 19:22:44,384 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backed error: java.lang.RuntimeException:
Unexpected data type 39 found in stream.
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.getGeneralizedDataType(BinInterSedes.java:1166)
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinInterSedesDatum(BinInterSedes.java:911)
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinSedesTuple(BinInterSedes.java:732)
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:695)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:78)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:70)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
Someone else is on the same situation?
Best regards,
--
Allan Avendaño S.
Computer Engineer
SWY22 Participant
GSOC 2012 Participant
Rome - Italy
Gmail: [EMAIL PROTECTED]
--
+
Allan 2012-10-06, 17:38
-
Re: Unexpected data type
Dmitriy Ryaboy 2012-10-07, 00:17
This is a serialization error.
What version of Pig are you running?
On Sat, Oct 6, 2012 at 10:38 AM, Allan <[EMAIL PROTECTED]> wrote:
> Hi everybody,
>
> I was trying to run a simple script:
>
> A = LOAD 'test01' AS (f1:chararray,f2:int,f3:chararray);
> B = order A by f1,f2,f3 DESC;
>
> but, suddenly I got this exception
>
> 2012-10-06 19:22:44,384 [main] ERROR
> org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> recreate exception from backed error: java.lang.RuntimeException:
> Unexpected data type 39 found in stream.
> at
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.getGeneralizedDataType(BinInterSedes.java:1166)
> at
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinInterSedesDatum(BinInterSedes.java:911)
> at
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinSedesTuple(BinInterSedes.java:732)
> at
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:695)
> at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:78)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
> at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:70)
> at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
> at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
> Someone else is on the same situation?
>
> Best regards,
>
> --
>
> Allan Avendaño S.
> Computer Engineer
> SWY22 Participant
> GSOC 2012 Participant
> Rome - Italy
> Gmail: [EMAIL PROTECTED]
> --
+
Dmitriy Ryaboy 2012-10-07, 00:17
-
Re: Unexpected data type
Allan 2012-10-07, 07:39
Hi Dmitriy,
Thanks for your reply, I'm working with a single node with hadoop version
1.0.0, I "checkout" three times Pig from trunk and I got the same exception.
What could I do to solve it?
On Sun, Oct 7, 2012 at 2:17 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
> This is a serialization error.
> What version of Pig are you running?
+
Allan 2012-10-07, 07:39
-
Re: Unexpected data type
Dmitriy Ryaboy 2012-10-07, 08:13
We saw this problem with order-by earlier in trunk, but I believe it
was fixed either this past week or the week before.
Which svn revision are you on?
D
On Sun, Oct 7, 2012 at 12:39 AM, Allan <[EMAIL PROTECTED]> wrote:
> Hi Dmitriy,
>
> Thanks for your reply, I'm working with a single node with hadoop version
> 1.0.0, I "checkout" three times Pig from trunk and I got the same exception.
> What could I do to solve it?
>
> On Sun, Oct 7, 2012 at 2:17 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
>
>> This is a serialization error.
>> What version of Pig are you running?
+
Dmitriy Ryaboy 2012-10-07, 08:13
-
Re: Unexpected data type
Allan 2012-10-07, 08:22
I'm working with svn revision 1395237.
On Sun, Oct 7, 2012 at 10:13 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:
> We saw this problem with order-by earlier in trunk, but I believe it
> was fixed either this past week or the week before.
> Which svn revision are you on?
>
+
Allan 2012-10-07, 08:22
-
Re: Unexpected data type
Jonathan Coveney 2012-10-07, 21:49
I would make sure you're running the newest version as there was indeed a
bug. If you're still getting the error, please ping the listserv again and
I will look into it.
2012/10/7 Allan <[EMAIL PROTECTED]>
> I'm working with svn revision 1395237.
>
> On Sun, Oct 7, 2012 at 10:13 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]>
> wrote:
>
> > We saw this problem with order-by earlier in trunk, but I believe it
> > was fixed either this past week or the week before.
> > Which svn revision are you on?
> >
>
+
Jonathan Coveney 2012-10-07, 21:49
-
Re: Unexpected data type
Allan 2012-10-08, 10:30
Updated to revision 1395503 and I still have this exception:
2012-10-08 12:29:52,754 [main] ERROR
org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
recreate exception from backed error: java.lang.RuntimeException:
Unexpected data type 39 found in stream.
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.getGeneralizedDataType(BinInterSedes.java:1166)
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinInterSedesDatum(BinInterSedes.java:911)
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinSedesTuple(BinInterSedes.java:732)
at
org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:695)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:78)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30)
at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83)
at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
at org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
This is the script that I'm using:
A = LOAD 'data4' AS
(name:chararray,surname:chararray,sales:double,code:int);
B = order A by name,surname,sales;
On Sun, Oct 7, 2012 at 11:49 PM, Jonathan Coveney <[EMAIL PROTECTED]>wrote:
> I would make sure you're running the newest version as there was indeed a
> bug. If you're still getting the error, please ping the listserv again and
> I will look into it.
+
Allan 2012-10-08, 10:30
-
Re: Unexpected data type
Jonathan Coveney 2012-10-08, 18:04
I just committed a fix and a test to trunk, reviewed by Julien. Please try
it out and let me know if you find any errors. Thanks for reporting this!
Jon
2012/10/8 Allan <[EMAIL PROTECTED]>
> Updated to revision 1395503 and I still have this exception:
>
> 2012-10-08 12:29:52,754 [main] ERROR
> org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable to
> recreate exception from backed error: java.lang.RuntimeException:
> Unexpected data type 39 found in stream.
> at
>
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.getGeneralizedDataType(BinInterSedes.java:1166)
> at
>
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinInterSedesDatum(BinInterSedes.java:911)
> at
>
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compareBinSedesTuple(BinInterSedes.java:732)
> at
>
> org.apache.pig.data.BinInterSedes$BinInterSedesTupleRawComparator.compare(BinInterSedes.java:695)
> at
>
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigSecondaryKeyComparator.compare(PigSecondaryKeyComparator.java:78)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.compare(MapTask.java:1111)
> at org.apache.hadoop.util.QuickSort.fix(QuickSort.java:30)
> at org.apache.hadoop.util.QuickSort.sortInternal(QuickSort.java:83)
> at org.apache.hadoop.util.QuickSort.sort(QuickSort.java:59)
> at
>
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.sortAndSpill(MapTask.java:1399)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1298)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:699)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:766)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
>
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
> This is the script that I'm using:
>
> A = LOAD 'data4' AS
> (name:chararray,surname:chararray,sales:double,code:int);
> B = order A by name,surname,sales;
>
> On Sun, Oct 7, 2012 at 11:49 PM, Jonathan Coveney <[EMAIL PROTECTED]
> >wrote:
>
> > I would make sure you're running the newest version as there was indeed a
> > bug. If you're still getting the error, please ping the listserv again
> and
> > I will look into it.
>
+
Jonathan Coveney 2012-10-08, 18:04
-
Re: Unexpected data type
Allan 2012-10-08, 22:13
I did some tests and I think it's working. Thanks!
I'll keep testing with rank operator.
On Mon, Oct 8, 2012 at 8:04 PM, Jonathan Coveney <[EMAIL PROTECTED]> wrote:
> I just committed a fix and a test to trunk, reviewed by Julien. Please try
> it out and let me know if you find any errors. Thanks for reporting this!
>
+
Allan 2012-10-08, 22:13
|
|
