Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Temporal in Hbase?


Copy link to this message
-
Re: Temporal in Hbase?
Shumin Wu 2012-10-10, 23:24
How I can miss this reply!!

Hi Anoop,

First, thanks for your reply to my question and apologize for not following
up promptly. I have put off a million of fires and come back to this issue.
Here are my thoughts. Yes, a FilterList with MUST_PASS_ALL works fine for
simple temporal clause.

However, I have a use case like this. I need to find all data having
overlapping time range for a given time range. Some data are valid till
now, which have a open-ended end time timestamp, marked as end_time = null
in our database.

To express it formally, for a given time range [const_st, const_end], where
const_st represents the constant start time and const_et the constant end
time, my task is to find all data rows with start_time and end_time
satisfying this expression:

start_time < const_et and end_time >= const_st or end_time is null.
In a FilterList, I can choose either MUST_PASS_ALL or MUST_PASS_ONE, but
none is applicable to this use case.

It would be nice if there is a temporal filter that allows me to select
data valid between [const_st, const_et] (and that end_time is null will be
automatically interpreted as valid up to now).

My domain is not traditionally Internet area, but I am sure folks in
clickstream business have a similar need. And I am wondering how they solve
this problem.

Temporal is commonly supported in traditional databases. So maybe HBase can
offer the same? I guess the current version does not have this support, and
a customer filter needs to be written by myself. I could be wrong. Please
enlighten me.
Shumin Wu

On Mon, Sep 17, 2012 at 8:16 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:

> Hi
> start_time and end_time are 2 qualifiers in your table.
> You can use a FilterList with MUST_PASS_ALL ( AND condition)
> Add SingleColumnValueFilter for each of the qualifier with the value and
> condition..
>
> -Anoop-
>
> ________________________________________
> From: Shumin Wu [[EMAIL PROTECTED]]
> Sent: Monday, September 17, 2012 9:58 PM
> To: [EMAIL PROTECTED]
> Subject: Temporal in Hbase?
>
> Hi,
>
> I have a user case to "filter" out rows using an "as of" predicate. For
> example, given a specific time point T, I would like to find all rows where
> start_time<=T<=end_time.
>
> Example hbase table schema:
> row_key, col_A, col_B, start_time, end_time
>
> I am wondering if there is any existing filter that allows me to do this.
>
> If not, I guess I would have to write my own custom filter.
>
> Thanks,
>
> Shumin Wu
>