Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill >> mail # dev >> Storage file format


Copy link to this message
-
Storage file format
Hi All,

I am interested in working on storage format. (sign up?)

I wrote a HDFS  file format, which is similar to Sequence file (row
storage, block management, compress), I provide InputFormat and
OutputFormat,

sometimes it get a great performance, sometimes not, depends on the data.

for Drill, we should implement a column-storage, this can skip some columns
during query, and skip some rows within one column file. but this
column-storage should based on the distributed file system, such as HDFS,
Mapr DFS, I like Mapr DFS because of HA.

we can implement the following column storage file format, I think it's
enough to us.

http://arxiv.org/pdf/1105.4252.pdf
+
moon soo Lee 2012-09-15, 12:47
+
Ted Dunning 2012-09-15, 13:44
+
NAVEEN MAANJU 2012-09-15, 13:54
+
Dharm Raj 2012-09-15, 15:09
+
Camuel Gilyadov 2012-09-15, 16:19
+
Dharm Raj 2012-09-15, 17:02
+
Tsuyoshi OZAWA 2012-09-15, 17:16
+
Ted Dunning 2012-09-15, 21:11
+
Azuryy Yu 2012-09-16, 00:07
+
Julien Le Dem 2012-09-16, 01:00
+
Tsuyoshi OZAWA 2012-09-19, 06:26
+
Ted Dunning 2012-09-15, 21:09
+
Lisen Mu 2013-04-07, 11:01