Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request 17557: [PIG-3456] Reduce threadlocal conf access in backend for each record


Copy link to this message
-
Re: Review Request 17557: [PIG-3456] Reduce threadlocal conf access in backend for each record

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17557/
-----------------------------------------------------------

(Updated Jan. 30, 2014, 8:37 p.m.)
Review request for pig.
Bugs: PIG-3456
    https://issues.apache.org/jira/browse/PIG-3456
Repository: pig
Description
-------

1) DefaultTuple has a protected boolean isNull = false; which is never used. Removed that.
2) Config checking with ThreadLocal conf is repeatedly done for each record. For eg: createDataBag in POCombinerPackage. But initialized only for first time in other places like POPackage, POJoinPackage, etc. PIG-3730 was one case which showed that config access was causing performance degradation. So replaced all occurrences with accessing config during initialization time.
Diffs (updated)
-----

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigGenericMapBase.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POCombinerPackage.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/PODistinct.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POLoad.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPackage.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POSort.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/Distinct.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/DefaultAbstractBag.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/DefaultTuple.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestTuple.java 1562947

Diff: https://reviews.apache.org/r/17557/diff/
Testing
-------

Ran full suite of unit tests
Thanks,

Rohini Palaniswamy