Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # dev - Review Request: when output hive table to file, users should could have a separator of their own choice


+
fangkun cao 2013-03-25, 08:33
Copy link to this message
-
Re: Review Request: when output hive table to file, users should could have a separator of their own choice
fangkun cao 2013-03-25, 09:32

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10115/
-----------------------------------------------------------

(Updated March 25, 2013, 9:32 a.m.)
Review request for hive.
Changes
-------

add test .q file
Description
-------

Type: New Feature
Priority: Minor  
Component/s: ql
Problem Definition?when output hive table to file,users should could have a separator of their own choice
Design:
Add a new command "insert overwrite [local] directory '...' [ROW FORMAT row_format] "
Examples :

insert overwrite local directory '${system:test.tmp.dir}/hive_test/multiins_local/src'
row format delimited
FIELDS TERMINATED BY ':'
LINES TERMINATED BY ','
select * from src ;

create table array_table (a array<string>, b array<string>)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
COLLECTION ITEMS TERMINATED BY ',';

load data local inpath "../data/files/array_table.txt" overwrite into table array_table;

insert overwrite local directory '${system:test.tmp.dir}/hive_test/multiins_local/array_table'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ':'
COLLECTION ITEMS TERMINATED BY '#'
select * from array_table;

CREATE TABLE map_table (foo STRING , bar MAP<STRING, STRING>)
ROW FORMAT DELIMITED
FIELDS TERMINATED BY '\t'
COLLECTION ITEMS TERMINATED BY ','
MAP KEYS TERMINATED BY ':'
STORED AS TEXTFILE;

load data local inpath "../data/files/map_table.txt" overwrite into table map_table;

insert overwrite local directory '${system:test.tmp.dir}/hive_test/multiins_local/map_table'
ROW FORMAT DELIMITED
FIELDS TERMINATED BY ':'
COLLECTION ITEMS TERMINATED BY '#'
MAP KEYS TERMINATED BY '='
select * from array_table;

Impact Analysis:
Test ideas?
This addresses bug HIVE-3682.
    https://issues.apache.org/jira/browse/HIVE-3682
Diffs (updated)
-----

  http://svn.apache.org/repos/asf/hive/trunk/data/files/array_table.txt PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/data/files/map_table.txt PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g 1460536
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/QB.java 1460536
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 1460536
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/LoadFileDesc.java 1460536
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 1460536
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/queries/clientpositive/insert_overwrite_local_directory_1.q PRE-CREATION
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/test/results/clientpositive/insert_overwrite_local_directory_1.q.out PRE-CREATION

Diff: https://reviews.apache.org/r/10115/diff/
Testing
-------
Thanks,

fangkun cao

+
fangkun cao 2013-03-25, 09:35