Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Review Request 17876: [PIG-3456] Reduce threadlocal conf access in backend for each record


Copy link to this message
-
Review Request 17876: [PIG-3456] Reduce threadlocal conf access in backend for each record

This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17876/

Review request for pig.
Bugs: PIG-3456
    https://issues.apache.org/jira/browse/PIG-3456
Repository: pig
Description

1) DefaultTuple has a protected boolean isNull = false; which is never used. Removing this gives ~3-5% improvement for big jobs
2) Config checking with ThreadLocal conf is repeatedly done for each record. For eg: createDataBag in POCombinerPackage. But initialized only for first time in other places like POPackage, POJoinPackage, etc.
Diffs

  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/mapReduceLayer/PigGenericMapBase.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POCombinerPackage.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/PODistinct.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POLoad.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPackage.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POPartialAgg.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/backend/hadoop/executionengine/physicalLayer/relationalOperators/POSort.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/builtin/Distinct.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/DefaultAbstractBag.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/src/org/apache/pig/data/DefaultTuple.java 1562947
  http://svn.apache.org/repos/asf/pig/trunk/test/org/apache/pig/test/TestTuple.java 1562947

Diff: https://reviews.apache.org/r/17876/diff/
Testing

Full suite of unit tests pass
Thanks,

Rohini Palaniswamy