Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Failed to get HCatalog to write to a Hive table using Pig


Copy link to this message
-
Failed to get HCatalog to write to a Hive table using Pig
lulynn_2008 2014-03-10, 02:55
Hi,
In pig grunt, when using HCatStorer to save data to hive table, following error happened. I am using pig-0.12.0, hive-0.12.0.  Do you have suggestions on how to trace the issues? Thanks
Here are the commands:
userinfo  = load '/user/biadmin/users/part-m-00000.avro' using org.apache.pig.piggybank.storage.avro.AvroStorage();
STORE userinfo INTO 'userinfo' USING org.apache.hcatalog.pig.HCatStorer();
Here are the errors:
ERROR [JobControl] org.apache.hadoop.security.UserGroupInformation     - PriviledgedActionException as:biadmin (auth:SIMPLE) cause:org.apache.hcatalog.common.HCatException : 2004 : HCatOutputFormat not initialized, setOutput has to be called
INFO  [JobControl] org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob     - PigLatin:users.pig got an error while submitting
org.apache.hcatalog.common.HCatException : 2004 : HCatOutputFormat not initialized, setOutput has to be called
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:111)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:97)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:85)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.checkOutputSpecs(HCatBaseOutputFormat.java:75)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecsHelper(PigOutputFormat.java:207)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:187)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1000)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:963)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:963)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:616)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
        at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
        at java.lang.Thread.run(Thread.java:738)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
WARN  [main] org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher     - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
ERROR [main] org.apache.pig.tools.pigstats.SimplePigStats     - ERROR 2997: Unable to recreate exception from backend error: org.apache.hcatalog.common.HCatException : 2004 : HCatOutputFormat not initialized, setOutput has to be called
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:111)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:97)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:85)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.checkOutputSpecs(HCatBaseOutputFormat.java:75)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecsHelper(PigOutputFormat.java:207)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:187)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1000)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:963)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:963)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:616)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
        at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
        at java.lang.Thread.run(Thread.java:738)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
ERROR [main] org.apache.pig.tools.pigstats.PigStatsUtil     - 1 map reduce job(s) failed!
ERROR [main] org.apache.pig.tools.grunt.GruntParser     - ERROR 2244: Job failed, hadoop does not return any error message