Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> viewfs on pig


Copy link to this message
-
Re: viewfs on pig
Hi,
Pig 0.11 should start supporting viewfs
https://issues.apache.org/jira/browse/PIG-2791

Johnny
On Tue, Apr 2, 2013 at 6:04 PM, 李建伟 <[EMAIL PROTECTED]> wrote:

> Hi,
>
>
> Do you know which version of Pig support Viewfs? I tried 0.11.0, got the
> following error:
> grunt> A = load '/tmp/passwd' using PigStorage(':');
> grunt> B = foreach A generate $0 as id;
> grunt> dump B;
> 2013-04-03 01:04:20,489 [main] INFO
>  org.apache.pig.tools.pigstats.ScriptState - Pig features used in the
> script: UNKNOWN
> 2013-04-03 01:04:20,608 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
> File concatenation threshold: 100 optimistic? false
> 2013-04-03 01:04:20,645 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size before optimization: 1
> 2013-04-03 01:04:20,646 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer
> - MR plan size after optimization: 1
> 2013-04-03 01:04:20,776 [main] WARN
>  org.apache.pig.backend.hadoop20.PigJobControl - falling back to default
> JobControl (not using hadoop 0.20 ?)
> java.lang.NoSuchFieldException: runnerState
>         at java.lang.Class.getDeclaredField(Class.java:1882)
>         at
> org.apache.pig.backend.hadoop20.PigJobControl.<clinit>(PigJobControl.java:51)
>         at
> org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.newJobControl(HadoopShims.java:97)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler.compile(JobControlCompiler.java:285)
>         at
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher.launchPig(MapReduceLauncher.java:177)
>         at org.apache.pig.PigServer.launchPlan(PigServer.java:1264)
>         at
> org.apache.pig.PigServer.executeCompiledLogicalPlan(PigServer.java:1249)
>         at org.apache.pig.PigServer.storeEx(PigServer.java:931)
>         at org.apache.pig.PigServer.store(PigServer.java:898)
>         at org.apache.pig.PigServer.openIterator(PigServer.java:811)
>         at
> org.apache.pig.tools.grunt.GruntParser.processDump(GruntParser.java:696)
>         at
> org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:320)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:194)
>         at
> org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:170)
>         at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:69)
>         at org.apache.pig.Main.run(Main.java:538)
>         at org.apache.pig.Main.main(Main.java:157)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>         at java.lang.reflect.Method.invoke(Method.java:597)
>         at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
> 2013-04-03 01:04:20,778 [main] INFO
>  org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added
> to the job
> 2013-04-03 01:04:20,792 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3
> 2013-04-03 01:04:20,798 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Using reducer estimator:
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
> 2013-04-03 01:04:20,800 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
> - BytesPerReducer=1000000000 maxReducers=999 totalInputFileSize=1513
> 2013-04-03 01:04:20,800 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Setting Parallelism to 1
> 2013-04-03 01:04:20,801 [main] INFO
>  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB