Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill >> mail # dev >> Storage file format


Copy link to this message
-
Storage file format
Hi All,

I am interested in working on storage format. (sign up?)

I wrote a HDFS  file format, which is similar to Sequence file (row
storage, block management, compress), I provide InputFormat and
OutputFormat,

sometimes it get a great performance, sometimes not, depends on the data.

for Drill, we should implement a column-storage, this can skip some columns
during query, and skip some rows within one column file. but this
column-storage should based on the distributed file system, such as HDFS,
Mapr DFS, I like Mapr DFS because of HA.

we can implement the following column storage file format, I think it's
enough to us.

http://arxiv.org/pdf/1105.4252.pdf
+
moon soo Lee 2012-09-15, 12:47
+
Ted Dunning 2012-09-15, 13:44
+
NAVEEN MAANJU 2012-09-15, 13:54
+
Dharm Raj 2012-09-15, 15:09
+
Camuel Gilyadov 2012-09-15, 16:19
+
Dharm Raj 2012-09-15, 17:02
+
Tsuyoshi OZAWA 2012-09-15, 17:16
+
Ted Dunning 2012-09-15, 21:11
+
Azuryy Yu 2012-09-16, 00:07
+
Julien Le Dem 2012-09-16, 01:00
+
Tsuyoshi OZAWA 2012-09-19, 06:26
+
Ted Dunning 2012-09-15, 21:09
+
Lisen Mu 2013-04-07, 11:01
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB