Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request 14056: HIVE-5246: Local task for map join submitted via oozie job fails on a secure HDFS


Copy link to this message
-
Review Request 14056: HIVE-5246: Local task for map join submitted via oozie job fails on a secure HDFS

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14056/
-----------------------------------------------------------

Review request for hive, Gunther Hagleitner and Thejas Nair.
Bugs: HIVE-5246
    https://issues.apache.org/jira/browse/HIVE-5246
Repository: hive-git
Description
-------

For a Hive query started by Oozie Hive action, the local task submitted for Mapjoin fails. The HDFS delegation token is not shared properly with the child JVM created for the local task.
Oozie creates a delegation token for the Hive action and sets env variable HADOOP_TOKEN_FILE_LOCATION as well as mapreduce.job.credentials.binary config property. However this doesn't get passed down to the child JVM which causes the problem.

This patch is modifying HIVE-4343 to load the token file path from env variable which will be set for both HiveServer2 (by original HIVE-4343 patch) as well as Oozie.
Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/exec/SecureCmdDoAs.java 8124639
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/ExecDriver.java 4cc7129
  ql/src/java/org/apache/hadoop/hive/ql/exec/mr/MapredLocalTask.java 341154d

Diff: https://reviews.apache.org/r/14056/diff/
Testing
-------

A test application attached to the ticket (https://issues.apache.org/jira/secure/attachment/12602266/HIVE-5246-test.tar) that simulates oozie behavior.
Manually tested HS2 to verify that it doesn't regress HIVE-4343
Thanks,

Prasad Mujumdar