Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # dev - Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files


Copy link to this message
-
Re: Review Request 17634: ZeroCopy and Direct ByteBuffer decompression for ORC files
Gopal V 2014-02-18, 06:36

This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17634/

(Updated Feb. 18, 2014, 6:36 a.m.)
Review request for hive, Gunther Hagleitner, Owen O'Malley, and Prasanth_J.
Changes

Removed 3 unused imports to hdfs-2 classes.
Bugs: HIVE-6347
    https://issues.apache.org/jira/browse/HIVE-6347
Repository: hive-git
Description

Zero Copy Read path for ORC + Zlib&Snappy decompressors wrapping the DirectDecompressor interface from HADOOP-10047
Diffs (updated)

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 86f1a71
  conf/hive-default.xml.template 8c64633
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/DirectDecompressionCodec.java PRE-CREATION
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/InStream.java 6da3d03
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/OrcInputFormat.java b9b4651
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/Reader.java 2bab0ce
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/ReaderImpl.java 56f25b7
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/RecordReaderImpl.java c3c9685
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/SnappyCodec.java e3131a3
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/VectorizedOrcInputFormat.java d904c44
  ql/src/java/org/apache/hadoop/hive/ql/io/orc/ZlibCodec.java a75fdea
  ql/src/test/org/apache/hadoop/hive/ql/io/orc/TestOrcFile.java 03fc705

Diff: https://reviews.apache.org/r/17634/diff/
Testing

Updated seek tests to test with zero-copy reads on.

TPC-DS queries on scale=2 flat ORC data.
Thanks,

Gopal V