Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Where are the rows in Trevni format?


+
sammefford 2013-03-21, 18:26
Copy link to this message
-
Re: Where are the rows in Trevni format?
Row numbers are not stored explicitly.  They are the implicit in the
ordinal position of values in the file.

Values are not sorted but are in row order.  The primary performance
advantage of a columnar file is that, when only a subset of columns
are required, only a subset of the data need be read.

Doug

On Thu, Mar 21, 2013 at 11:26 AM, sammefford <[EMAIL PROTECTED]> wrote:
> I read the Trevni Specificaiton:
> http://avro.apache.org/docs/1.7.4/trevni/spec.html
> and I can't see where the row ids are stored for each value in each column.
> Am I missing something obvious?  Is the spec incomplete on that point?
>
> Also, to confirm, my understanding is columnar formats are efficient because
> they store column values sorted and can thereby find specific values or
> ranges of values quickly.  While the spec mentions the benefits of sorting,
> I don't see a requirement that column values be sorted.  Can we depend that
> the blocks of column values are sorted?
>
> Thanks,
>
> Sam Mefford
> Chief Architect-Big Data Solutions
> Avalon Consluting, LLC.
> 801-706-9731
>
>
>
> --
> View this message in context: http://apache-avro.679487.n3.nabble.com/Where-are-the-rows-in-Trevni-format-tp4026663.html
> Sent from the Avro - Users mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB