Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Pig 0.10.0 and Hadoop 0.23


+
Johannes Schwenk 2012-06-20, 11:38
+
Daniel Dai 2012-06-20, 18:21
+
Johannes Schwenk 2012-06-21, 08:16
+
Daniel Dai 2012-06-27, 20:24
Copy link to this message
-
Re: Pig 0.10.0 and Hadoop 0.23
Johannes Schwenk 2012-07-03, 16:12
Thanks.

Well, I did that and rebuild all pig jars with -Dhadoopversion=23 but am
still getting errors - this time around it's errors with opening an
iterator on alias b again, sigh...

I have singled out one of the similarly failing tests and pasted the output:

http://pastebin.com/reKjcX0p

The test case failing looks like this:

http://pastebin.com/L5seh0ka

And INPUT_FILE gets created correctly through @BeforeClass

http://pastebin.com/wKrH7i50

If I run the script in the grunt shell invoked like this

pig -x local -propertyFile
/home/schwenk/dev/hadoop/trunk/pig/conf/adition.properties

everything works fine and I can execute the script

grunt> a = LOAD 'input.dat' AS (id:int, grp:int, additional:int,
referer:chararray);
b = FILTER a BY com.adition.pig.filtering.string.CONTAINS(referer, 'obama');
grunt> dump b;
...
2012-07-03 18:11:23,032 [main] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
input paths to process : 1
(4,323,242,http://www.google.com/url&url=http%3A%2F%2Fwww.tagesschau.de&q=barack%20obama)
(5,423,342,http://www.google.com/url&url=http%3A%2F%2Fwww.bild.de&q=obama)
and verify that the output is correct.

Any further advice on this?

Thanks,
Johannes

Am 27.06.2012 22:24, schrieb Daniel Dai:
> Do "ant clean".
>
> On Thu, Jun 21, 2012 at 1:16 AM, Johannes Schwenk <
> [EMAIL PROTECTED]> wrote:
>
>> Am 20.06.2012 20:21, schrieb Daniel Dai:
>>> Have you recompiled Pig using -Dhadoopversion=23?
>>
>> No, thanks that solved the issue at hand!
>>
>> I have other errors though. My test cases return errors like
>>
>> http://pastebin.com/XpszYeu2
>>
>> So I am still doing something wrong it seems?!
>>
>> Thanks,
>> Johannes
>>
>>
>>> On Wed, Jun 20, 2012 at 4:38 AM, Johannes Schwenk <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi list,
>>>>
>>>> I have written two UDFs that run fine with Hadoop 0.20.2 and Pig 0.10.0.
>>>> I am trying to switch from Hadoop 0.20.2 to 0.23 and am encountering
>>>> some errors on running my tests:
>>>>
>>>> http://pastebin.com/JwHRQ9RQ
>>>>
>>>> Might this be a simple error in configuration?
>>>>
>>>> Johannes Schwenk
>>>>
>>>> --
>>>> Softwareentwickler (Reporting)
>>>> ________________________________________________________
>>>>
>>>> ADITION technologies AG
>>>> Schwarzwaldstraße 78b
>>>> 79117 Freiburg
>>>>
>>>> http://www.adition.com
>>>>
>>>> T +49 / (0)761 / 88147 - 30
>>>> F +49 / (0)761 / 88147 - 77
>>>> SUPPORT +49  / (0)1805 - ADITION
>>>>
>>>> (Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)
>>>>
>>>> Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
>>>> Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus
>> Schlüter
>>>> Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer
>>>> UStIDNr.: DE 218 858 434
>>>>
>>>>
>>>
>>
>>
>>
>> Johannes Schwenk
>>
>> --
>> Softwareentwickler (Reporting)
>> ________________________________________________________
>>
>> ADITION technologies AG
>> Schwarzwaldstraße 78b
>> 79117 Freiburg
>>
>> http://www.adition.com
>>
>> T +49 / (0)761 / 88147 - 30
>> F +49 / (0)761 / 88147 - 77
>> SUPPORT +49  / (0)1805 - ADITION
>>
>> (Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)
>>
>> Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
>> Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus Schlüter
>> Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer
>> UStIDNr.: DE 218 858 434
>>
>>
>

Johannes Schwenk

--
Softwareentwickler (Reporting)
________________________________________________________

ADITION technologies AG
Schwarzwaldstraße 78b
79117 Freiburg

http://www.adition.com

T +49 / (0)761 / 88147 - 30
F +49 / (0)761 / 88147 - 77
SUPPORT +49  / (0)1805 - ADITION

(Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)

Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus Schlüter
Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer
UStIDNr.: DE 218 858 434