Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files


Copy link to this message
-
Re: Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files
Gopal V 2014-02-26, 07:28

This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17634/

(Updated Feb. 26, 2014, 7:28 a.m.)
Review request for hive, Gunther Hagleitner, Owen O'Malley, and Prasanth_J.
Changes

Rebase to trunk API
Bugs: HIVE-6347
    https://issues.apache.org/jira/browse/HIVE-6347
Repository: hive-git
Description

Zero Copy Read path for ORC + Zlib&Snappy decompressors wrapping the DirectDecompressor interface from HADOOP-10047
Diffs (updated)

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 0f3ef7c
  conf/hive-default.xml.template 1feb9cb
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/DirectDecompressionCodec.java PRE-CREATION
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/InStream.java 6da3d03
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/MemoryManager.java 9af12de
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java 7798a7c
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/SnappyCodec.java e3131a3
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java 27a9338
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/ZlibCodec.java a75fdea
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcFile.java 4d3013d
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestStringRedBlackTree.java 251cd83

Diff: https://reviews.apache.org/r/17634/diff/
Testing

Updated seek tests to test with zero-copy reads on.

TPC-DS queries on scale=2 flat ORC data.
Thanks,

Gopal V