Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> PK violation during Hive add partition


Copy link to this message
-
PK violation during Hive add partition
Hello,

We are running into intermittent errors while running the below query.  Some background on this, our table (tbl_someTable) that we're altering is an external table, and the query below is run concurrently by multiple oozie workflows.

ALTER TABLE tbl_someTable ADD IF NOT EXISTS PARTITION(cluster_address = '${CLUSTERADDRESS}', upload_date = '${PREVIOUSDATE}' , upload_hour = '${PREVIOUSHOUR}')
LOCATION 'asv://${RAWLOGSCONTAINER}/${CLUSTERADDRESS}/someLog/${PREVIOUSDATE}/${PREVIOUSHOUR}';

The errors we're getting are below.

Is this a known issue and is there a workaround for it?

Thanks
karlen

stderr logs
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in jar:file:/c:/hdfs/mapred/local/taskTracker/distcache/5662320028645753518_889604055_1925270295/10.175.202.81/user/dssxuser/share/lib/hive/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history file=/tmp/dssxuser/hive_job_log_dssxuser_201212070113_1149932084.txt
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MPartition@2a4e50f<mailto:org.apache.hadoop.hive.metastore.model.MPartition@2a4e50f>" using statement "INSERT INTO PARTITIONS (PART_ID,CREATE_TIME,SD_ID,PART_NAME,LAST_ACCESS_TIME,TBL_ID) VALUES (?,?,?,?,?,?)" failed : Violation of PRIMARY KEY constraint 'PK_partitions_PART_ID'. Cannot insert duplicate key in object 'dbo.PARTITIONS'. The duplicate key value is (221).
NestedThrowables:
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of PRIMARY KEY constraint 'PK_partitions_PART_ID'. Cannot insert duplicate key in object 'dbo.PARTITIONS'. The duplicate key value is (221).
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Intercepting System.exit(9)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [9]
stderr logs
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Logging initialized using configuration in jar:file:/c:/hdfs/mapred/local/taskTracker/distcache/2751940372978647467_889604055_1925270295/10.175.202.81/user/dssxuser/share/lib/hive/hive-common-0.9.0.jar!/hive-log4j.properties
Hive history file=/tmp/dssxuser/hive_job_log_dssxuser_201212071515_173032638.txt
FAILED: Error in metadata: javax.jdo.JDODataStoreException: Insert of object "org.apache.hadoop.hive.metastore.model.MSerDeInfo@31ce40d5<mailto:org.apache.hadoop.hive.metastore.model.MSerDeInfo@31ce40d5>" using statement "INSERT INTO SERDES (SERDE_ID,SLIB,"NAME") VALUES (?,?,?)" failed : Violation of PRIMARY KEY constraint 'PK_serdes_SERDE_ID'. Cannot insert duplicate key in object 'dbo.SERDES'. The duplicate key value is (2006).
NestedThrowables:
com.microsoft.sqlserver.jdbc.SQLServerException: Violation of PRIMARY KEY constraint 'PK_serdes_SERDE_ID'. Cannot insert duplicate key in object 'dbo.SERDES'. The duplicate key value is (2006).
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
Intercepting System.exit(9)
Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.HiveMain], exit code [9]
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB