Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Pig Schema contains a name that is not allowed in Avro


+
Johannes Schwenk 2013-10-21, 17:05
+
j.barrett Strausser 2013-10-21, 17:29
Copy link to this message
-
Re: Pig Schema contains a name that is not allowed in Avro
Johannes Schwenk 2013-10-22, 08:16
Thanks for your answer!

Actually the Avro schema is valid and I can load data with it. The error
message states, that pig has a problem with the Pig schema, which has no
duplicate names.

Johannes

Am 21.10.2013 19:29, schrieb j.barrett Strausser:
> I'd imagime it is having an issue with the duplicate 'd' names? That is my
> un-informed guess though.
>
>
> On Mon, Oct 21, 2013 at 1:05 PM, Johannes Schwenk <
> [EMAIL PROTECTED]> wrote:
>
>> Hi!
>>
>> I'm getting the following error when running the script below in grunt
>> (pig 0.12.0):
>>
>> DEFINE AvroStorageSTORE AvroStorage('-f schema.avsc');
>> data = LOAD '/path/to/data' AS(a:long, b:long, c:int, d:chararray,
>> e:int, f:int, g:int, h:long, i:chararray, j:int, k:int, l:int, m:int,
>> n:int, o:int, p:int, q:int, r:int, s:int, t:int, u:int, v:chararray,
>> w:chararray, x:chararray);
>> STORE data into '/path/to/output' USING AvroStorageSTORE();
>>
>> 2013-10-21 18:50:15,491 [main] INFO
>> org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
>> script: UNKNOWN
>> 2013-10-21 18:50:15,554 [main] ERROR org.apache.pig.tools.grunt.Grunt -
>> ERROR 2116:
>> <line 3, column 0> Output Location Validation Failed for:
>> 'hdfs://path/to/output More info to follow:
>> Pig Schema contains a name that is not allowed in Avro
>> Details at logfile: pig_1382374188771.log
>>
>> Logfile contains:
>>
>> Pig Schema contains a name that is not allowed in Avro
>>
>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable
>> to store alias data
>>         at
>> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1596)
>>         at org.apache.pig.PigServer.registerQuery(PigServer.java:584)
>>         at
>> org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:967)
>>         at
>>
>> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:386)
>>         at
>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:189)
>>         at
>>
>> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:165)
>>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>>         at org.apache.pig.Main.run(Main.java:495)
>>         at org.apache.pig.Main.main(Main.java:111)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>>
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>>
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
>> Caused by: org.apache.pig.impl.plan.VisitorException: ERROR 2116:
>> <line 3, column 0> Output Location Validation Failed for:
>> 'hdfs://path/to/output More info to follow:
>> Pig Schema contains a name that is not allowed in Avro
>>         at
>>
>> org.apache.pig.newplan.logical.rules.InputOutputFileValidator$InputOutputFileVisitor.visit(InputOutputFileValidator.java:75)
>>         at
>> org.apache.pig.newplan.logical.relational.LOStore.accept(LOStore.java:77)
>>         at
>>
>> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:64)
>>         at
>>
>> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
>>         at
>>
>> org.apache.pig.newplan.DepthFirstWalker.depthFirst(DepthFirstWalker.java:66)
>>         at
>> org.apache.pig.newplan.DepthFirstWalker.walk(DepthFirstWalker.java:53)
>>         at org.apache.pig.newplan.PlanVisitor.visit(PlanVisitor.java:50)
>>         at
>>
>> org.apache.pig.newplan.logical.rules.InputOutputFileValidator.validate(InputOutputFileValidator.java:45)
>>         at
>>
>> org.apache.pig.backend.hadoop.executionengine.HExecutionEngine.compile(HExecutionEngine.java:294)
>>         at org.apache.pig.PigServer.compilePp(PigServer.java:1360)
>>         at
>> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1297)

Johannes Schwenk

Softwareentwickler (Reporting)
________________________________________________________

ADITION technologies AG
Schwarzwaldstraße 78b
79117 Freiburg

http://www.adition.com

T +49 / (0)761 / 88147 - 30
F +49 / (0)761 / 88147 - 77
SUPPORT +49  / (0)1805 - ADITION

(Festnetzpreis 14 ct/min; Mobilfunkpreise maximal 42 ct/min)

Eingetragen beim Amtsgericht Düsseldorf unter HRB 54076
Vorstände: Andreas Kleiser, Jörg Klekamp, Tihomir Perkovic, Marcus Schlüter
Aufsichtsratsvorsitzender: Joachim Schneidmadl
UStIDNr.: DE 218 858 434
+
Ruslan Al-Fakikh 2013-11-19, 16:40