Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Failed to get HCatalog to write to a Hive table using Pig


Copy link to this message
-
Failed to get HCatalog to write to a Hive table using Pig
Hi,
In pig grunt, when using HCatStorer to save data to hive table, following error happened. I am using pig-0.12.0, hive-0.12.0.  Do you have suggestions on how to trace the issues? Thanks
Here are the commands:
userinfo  = load '/user/biadmin/users/part-m-00000.avro' using org.apache.pig.piggybank.storage.avro.AvroStorage();
STORE userinfo INTO 'userinfo' USING org.apache.hcatalog.pig.HCatStorer();
Here are the errors:
ERROR [JobControl] org.apache.hadoop.security.UserGroupInformation     - PriviledgedActionException as:biadmin (auth:SIMPLE) cause:org.apache.hcatalog.common.HCatException : 2004 : HCatOutputFormat not initialized, setOutput has to be called
INFO  [JobControl] org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob     - PigLatin:users.pig got an error while submitting
org.apache.hcatalog.common.HCatException : 2004 : HCatOutputFormat not initialized, setOutput has to be called
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:111)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:97)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:85)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.checkOutputSpecs(HCatBaseOutputFormat.java:75)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecsHelper(PigOutputFormat.java:207)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:187)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1000)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:963)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:963)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:616)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
        at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
        at java.lang.Thread.run(Thread.java:738)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
WARN  [main] org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher     - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to stop immediately on failure.
ERROR [main] org.apache.pig.tools.pigstats.SimplePigStats     - ERROR 2997: Unable to recreate exception from backend error: org.apache.hcatalog.common.HCatException : 2004 : HCatOutputFormat not initialized, setOutput has to be called
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:111)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getJobInfo(HCatBaseOutputFormat.java:97)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.getOutputFormat(HCatBaseOutputFormat.java:85)
        at org.apache.hcatalog.mapreduce.HCatBaseOutputFormat.checkOutputSpecs(HCatBaseOutputFormat.java:75)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecsHelper(PigOutputFormat.java:207)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat.checkOutputSpecs(PigOutputFormat.java:187)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:1000)
        at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:963)
        at java.security.AccessController.doPrivileged(AccessController.java:310)
        at javax.security.auth.Subject.doAs(Subject.java:573)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1502)
        at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:963)
        at org.apache.hadoop.mapreduce.Job.submit(Job.java:616)
        at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:336)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:60)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37)
        at java.lang.reflect.Method.invoke(Method.java:611)
        at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
        at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:191)
        at java.lang.Thread.run(Thread.java:738)
        at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:270)
ERROR [main] org.apache.pig.tools.pigstats.PigStatsUtil     - 1 map reduce job(s) failed!
ERROR [main] org.apache.pig.tools.grunt.GruntParser     - ERROR 2244: Job failed, hadoop does not return any error message
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB