-Notes from HBase Types/Serialization Discussion
Stack 2013-03-07, 22:16
(I'm sending this for Doug Meil; he is having difficulty getting this up on
the dev list)
Notes from an HBase client conference call a few of us had yesterday.
Attendees: Enis, Nick D., James Taylor, Stack, LarsH, Doug Meil and Andy
- Discussion of Serialization Scheme Independent of Java
- After a short discussion, this is a very interesting idea but off
the table for the next 4-6 releases.
- E.g., would we have to invent serialization for C++ or Python
clients? It's not a small undertaking.
- Nobody was aware of any other open source libraries that did this.
- Component Priority
- After a healthy discussion, the following priority emerged:
- 1) Serialized Types
- But focusing on Java. Using some combination of Orderly and
- Nick/James to lead this.
- New Jira will be created for this effort, with sub-tasks for
- 2) Utilities
- E.g., something akin to what HBASE-7221 was trying to achieve.
This might also be a documentation exercise on how to
properly use Orderly
- I.e., utilities to build rowkeys (et al.) in code for use by
applications (I.e., not Hbase itself)
- Consensus on support for both FixedLength and VariableLength
rowkeys, to allow for faster scans especially in the former case. But
still providing utilities for variable length keys because
folks still use
things like URLs in rowkeys.
- 3) Schema
- Schema definitions of rowkeys and tables that could be
externalized to Hive, et al.
- HBase-Centric API
- With respect to Utilities and Schemas, there was consensus that
completely enveloping HTable is not the suggested path, but rather we
should look for opportunities to improve the Get, Put, Scan
Htable but make it less tedious with byte-array handling.
- This will be closed as "won't fixed", but will be linked from
whatever the new Jira is as a related ticket.
- Agreeing To Something And Actually Doing It
- Stack proposed the novel idea of all of us agreeing to something
and then actually doing it, and there was consensus that this was a good
Andrew Purtell 2013-03-08, 01:33
Doug Meil 2013-03-07, 22:21