Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> viewfs on pig


Copy link to this message
-
Re: viewfs on pig
Hi,
Pig 0.11 should start supporting viewfs
https://issues.apache.org/jira/browse/PIG-2791

Johnny
On Tue, Apr 2, 2013 at 6:04 PM, 李建伟 <[EMAIL PROTECTED]> wrote:

> Hi,
>
>
> Do you know which version of Pig support Viewfs? I tried 0.11.0, got the
> following error:
> grunt> A = load '/tmp/passwd' using PigStorage(':');
> grunt> B = foreach A generate $0 as id;
> grunt> dump B;
> 2013-04-03 01:04:20,489 [main] INFO
>  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2013-04-03 01:04:20,608 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2013-04-03 01:04:20,645 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2013-04-03 01:04:20,646 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2013-04-03 01:04:20,776 [main] WARN
>  org.apache.pig.backend.hadoop20.PigJobControl - falling back to default
> JobControl (not using hadoop 0.20 ?)
> java.lang.NoSuchFieldException: runnerState
>         at java.lang.Class.getDeclaredField(Class.java:1882)
>         at
> org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
>         at
> org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:97)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:285)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
>         at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
>         at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
>         at org.apache.pig.PigServer.storeEx(PigServer.java:931)
>         at org.apache.pig.PigServer.store(PigServer.java:898)
>         at org.apache.pig.PigServer.openIterator(PigServer.java:811)
>         at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>         at org.apache.pig.Main.run(Main.java:538)
>         at org.apache.pig.Main.main(Main.java:157)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> 2013-04-03 01:04:20,778 [main] INFO
>  org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> to the job
> 2013-04-03 01:04:20,792 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2013-04-03 01:04:20,798 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Using reducer estimator:
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
> 2013-04-03 01:04:20,800 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
> - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=1513
> 2013-04-03 01:04:20,800 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting Parallelism to 1
> 2013-04-03 01:04:20,801 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler