Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Error running PIG 12 with Hadoop 0.23.1.


Copy link to this message
-
Error running PIG 12 with Hadoop 0.23.1.
Hi All,

I am trying to run PIG 12 with Hadoop 0.23.1 and getting following error
msg, Can someone please help and suggest what I am missing.  I can run PIG
in local mode without any issue which executes with inbuilt Hadoop.  I had
recompiled PIG with hadoop 23, but so far no luck in resolving this issue.

*grunt> A = load 'NYSE_dividends' as (exch, symb, dt, div);*
*grunt> dump A ;*
...
...
*2013-11-26 13:20:35,975 [JobControl] ERROR
org.apache.pig.backend.hadoop23.PigJobControl - Error while trying to run
jobs.java.lang.RuntimeException:
java.lang.reflect.InvocationTargetException*
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(
PigJobControl.java:130)
        at org.apache.pig.backend.hadoop23.PigJobControl.run(
PigJobControl.java:191)
        at java.lang.Thread.run(Thread.java:619)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
MapReduceLauncher$1.run(MapReduceLauncher.java:270)
*Caused by: java.lang.reflect.InvocationTargetException        at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)*
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(
PigJobControl.java:128)
        ... 3 more
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.fs.FileSystem.
getDefaultBlockSize(Lorg/apache/hadoop/fs/Path;)J
        at org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.
getDefaultBlockSize(HadoopShims.java:108)
        at org.apache.pig.backend.hadoop.executionengine.
mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:277)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(
JobSubmitter.java:451)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(
JobSubmitter.java:468)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(
JobSubmitter.java:360)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1221)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1177)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1218)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.
submit(ControlledJob.java:336)
        ... 8 more
2013-11-26 13:20:35,976 [main] INFO  org.apache.pig.backend.hadoop.
executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId:
job_1385489868206_0001

--
Regards,
Uttam Kumar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB