Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Pig 0.10.0 and Hadoop 0.23


+
Johannes Schwenk 2012-06-20, 11:38
+
Daniel Dai 2012-06-20, 18:21
+
Johannes Schwenk 2012-06-21, 08:16
+
Daniel Dai 2012-06-27, 20:24
Copy link to this message
-
Re: Pig 0.10.0 and Hadoop 0.23
Thanks.

Well, I did that and rebuild all pig jars with -Dhadoopversion=23 but am
still getting errors - this time around it's errors with opening an
iterator on alias b again, sigh...

I have singled out one of the similarly failing tests and pasted the output:

http://pastebin.com/reKjcX0p

The test case failing looks like this:

http://pastebin.com/L5seh0ka

And INPUT_FILE gets created correctly through @BeforeClass

http://pastebin.com/wKrH7i50

If I run the script in the grunt shell invoked like this

pig -x local -propertyFile
/home/schwenk/dev/hadoop/trunk/pig/conf/adition.properties

everything works fine and I can execute the script

grunt> a = LOAD 'input.dat' AS (id:int, grp:int, additional:int,
referer:chararray);
b = FILTER a BY com.adition.pig.filtering.string.CONTAINS(referer, 'obama');
grunt> dump b;
...
2012-07-03 18:11:23,032 [main] INFO
org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total
input paths to process : 1
(4,323,242,http://www.google.com/url&url=http%3A%2F%2Fwww.tagesschau.de&q=barack%20obama)
(5,423,342,http://www.google.com/url&url=http%3A%2F%2Fwww.bild.de&q=obama)
and verify that the output is correct.

Any further advice on this?

Thanks,
Johannes

Am 27.06.2012 22:24, schrieb Daniel Dai:
> Do "ant clean".
>
> On Thu, Jun 21, 2012 at 1:16 AM, Johannes Schwenk <
> [EMAIL PROTECTED]> wrote:
>
>> Am 20.06.2012 20:21, schrieb Daniel Dai:
>>> Have you recompiled Pig using -Dhadoopversion=23?
>>
>> No, thanks that solved the issue at hand!
>>
>> I have other errors though. My test cases return errors like
>>
>> http://pastebin.com/XpszYeu2
>>
>> So I am still doing something wrong it seems?!
>>
>> Thanks,
>> Johannes
>>
>>
>>> On Wed, Jun 20, 2012 at 4:38 AM, Johannes Schwenk <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi list,
>>>>
>>>> I have written two UDFs that run fine with Hadoop 0.20.2 and Pig 0.10.0.
>>>> I am trying to switch from Hadoop 0.20.2 to 0.23 and am encountering
>>>> some errors on running my tests:
>>>>
>>>> http://pastebin.com/JwHRQ9RQ
>>>>
>>>> Might this be a simple error in configuration?
>>>>
>>>> Johannes Schwenk
>>>>
>>>> --
>>>> Softwareentwickler (Reporting)
>>>> ________________________________________________________
>>>>
>>>> ADITION technologies AG
>>>> Schwarzwaldstraße 78b
>>>> 79117 Freiburg
>>>>
>>>> http://www.adition.com
>>>>
>>>> T +49 / (0)761 / 88147 - 30
>>>> F +49 / (0)761 / 88147 - 77
>>>> SUPPORT +49  / (0)1805 - ADITION
>>>>
>>>> (Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)
>>>>
>>>> Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
>>>> Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus
>> Schlüter
>>>> Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer
>>>> UStIDNr.: DE 218 858 434
>>>>
>>>>
>>>
>>
>>
>>
>> Johannes Schwenk
>>
>> --
>> Softwareentwickler (Reporting)
>> ________________________________________________________
>>
>> ADITION technologies AG
>> Schwarzwaldstraße 78b
>> 79117 Freiburg
>>
>> http://www.adition.com
>>
>> T +49 / (0)761 / 88147 - 30
>> F +49 / (0)761 / 88147 - 77
>> SUPPORT +49  / (0)1805 - ADITION
>>
>> (Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)
>>
>> Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
>> Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus Schlüter
>> Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer
>> UStIDNr.: DE 218 858 434
>>
>>
>

Johannes Schwenk

--
Softwareentwickler (Reporting)
________________________________________________________

ADITION technologies AG
Schwarzwaldstraße 78b
79117 Freiburg

http://www.adition.com

T +49 / (0)761 / 88147 - 30
F +49 / (0)761 / 88147 - 77
SUPPORT +49  / (0)1805 - ADITION

(Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)

Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus Schlüter
Aufsichtsratsvorsitzender: Rechtsanwalt Daniel Raimer
UStIDNr.: DE 218 858 434
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB