Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Error running PIG 12 with Hadoop 0.23.1.


Copy link to this message
-
Error running PIG 12 with Hadoop 0.23.1.
Uttam Kumar 2013-12-02, 19:26
Hi All,

I am trying to run PIG 12 with Hadoop 0.23.1 and getting following error
msg, Can someone please help and suggest what I am missing.  I can run PIG
in local mode without any issue which executes with inbuilt Hadoop.  I had
recompiled PIG with hadoop 23, but so far no luck in resolving this issue.

*grunt> A = load 'NYSE_dividends' as (exch, symb, dt, div);*
*grunt> dump A ;*
...
...
*2013-11-26 13:20:35,975 [JobControl] ERROR
org.apache.pig.backend.hadoop23.PigJobControl - Error while trying to run
jobs.java.lang.RuntimeException:
java.lang.reflect.InvocationTargetException*
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(
PigJobControl.java:130)
        at org.apache.pig.backend.hadoop23.PigJobControl.run(
PigJobControl.java:191)
        at java.lang.Thread.run(Thread.java:619)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
MapReduceLauncher$1.run(MapReduceLauncher.java:270)
*Caused by: java.lang.reflect.InvocationTargetException        at
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)        at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)*
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(
DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(
PigJobControl.java:128)
        ... 3 more
Caused by: java.lang.NoSuchMethodError: org.apache.hadoop.fs.FileSystem.
getDefaultBlockSize(Lorg/apache/hadoop/fs/Path;)J
        at org.apache.pig.backend.hadoop.executionengine.shims.HadoopShims.
getDefaultBlockSize(HadoopShims.java:108)
        at org.apache.pig.backend.hadoop.executionengine.
mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:277)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(
JobSubmitter.java:451)
        at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(
JobSubmitter.java:468)
        at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(
JobSubmitter.java:360)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1221)
        at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1218)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(
UserGroupInformation.java:1177)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:1218)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.
submit(ControlledJob.java:336)
        ... 8 more
2013-11-26 13:20:35,976 [main] INFO  org.apache.pig.backend.hadoop.
executionengine.mapReduceLayer.MapReduceLauncher - HadoopJobId:
job_1385489868206_0001

--
Regards,
Uttam Kumar
+
Rohini Palaniswamy 2013-12-06, 20:25
+
Uttam Kumar 2013-12-09, 20:43