Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request 15502: PIG-3525 PigStats.get() and ScriptState.get() shouldn't return MR-specific objects


Copy link to this message
-
Review Request 15502: PIG-3525 PigStats.get() and ScriptState.get() shouldn't return MR-specific objects

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/15502/
-----------------------------------------------------------

Review request for pig, Aniket Mokashi, Daniel Dai, and Rohini Palaniswamy.
Bugs: PIG-3525
    https://issues.apache.org/jira/browse/PIG-3525
Repository: pig-git
Description
-------

PigStats and ScriptState are thread local variables that are accessible via PigStats#get() and ScriptState#get(). Currently, it's possible that these get() functions return MR-specific objects regardless of the exec type.

The patch fixes the problem as follows-
* ExecutionEngine interface provides two methods instantiatePigStats() and instantiateScriptState(), and they can be used to instantiate the right type of PigStats and ScriptState objects.
* PigStats#get() and ScriptState#get() make calls to instantiatePigStats() and instantiateScriptState() to initialize thread local variables at runtime.
* PigStats#start(PigStats) and ScriptState#start(ScriptState) are used to explicitly reset thread local variables with new objects.

Note that the PigContext field in PigStats and ScriptState classes is changed to a static thread local variable because it is accessed inside static functions PigStats.get() and ScriptState.get(). I chose this approach over passing a PigContext parameter to get() functions because the latter involves more changes due to the function signature changes.
Diffs
-----

  src/org/apache/pig/Main.java 9ba1b86
  src/org/apache/pig/backend/executionengine/ExecutionEngine.java 56a958b
  src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/MRExecutionEngine.java ddc89f9
  src/org/apache/pig/scripting/BoundScript.java 28340d1
  src/org/apache/pig/tools/pigstats/PigStats.java ff0d0f9
  src/org/apache/pig/tools/pigstats/PigStatsUtil.java 84c52de
  src/org/apache/pig/tools/pigstats/ScriptState.java e3a2ba2
  src/org/apache/pig/tools/pigstats/mapreduce/SimplePigStats.java bd35405

Diff: https://reviews.apache.org/r/15502/diff/
Testing
-------

ant clean test-commit.
Thanks,

Cheolsoo Park