Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Temporal in Hbase?


+
Shumin Wu 2012-09-17, 16:28
+
Anoop Sam John 2012-09-18, 03:16
+
Shumin Wu 2012-10-10, 23:24
+
Ramkrishna.S.Vasudevan 2012-10-11, 05:32
Copy link to this message
-
RE: Temporal in Hbase?
Hi Shumin,

>start_time < const_et and end_time >= const_st or end_time is null.
Your problem was end_time >= const_st or end_time is null.....

You can make use of FilterList with MUST_PASS_ALL (AND) condition only.. This can contain one SingleColumnValueFilter correspodning to start_time with your condition and value.  "end_time" is null means you are not having any KV added in the row for this column? [This column value is missing for a row] In that case you can use a SingleColumnValueFilter with condition and value "const_st"  and to this filter set  filterIfMissing(false). This means if the column is absent for a row, that row wont get filtered out.. This is what you want right? [But the default value for filterIfMissing is false only ]

FYI a FilterList can contain another filter list
So if you have a query like  col1=? AND ( col2=? OR col2=? ) you can use FilterList.. One inner filter list with MUST_PASS_ONE for col2 and an outer FL with MUST_PASS_ALL which contains the inner FL and SCVF for col1..    Hope I understood your problem and giving the answer which you are looking for   :)

-Anoop-
________________________________________
From: Shumin Wu [[EMAIL PROTECTED]]
Sent: Thursday, October 11, 2012 4:54 AM
To: [EMAIL PROTECTED]
Subject: Re: Temporal in Hbase?

How I can miss this reply!!

Hi Anoop,

First, thanks for your reply to my question and apologize for not following
up promptly. I have put off a million of fires and come back to this issue.
Here are my thoughts. Yes, a FilterList with MUST_PASS_ALL works fine for
simple temporal clause.

However, I have a use case like this. I need to find all data having
overlapping time range for a given time range. Some data are valid till
now, which have a open-ended end time timestamp, marked as end_time = null
in our database.

To express it formally, for a given time range [const_st, const_end], where
const_st represents the constant start time and const_et the constant end
time, my task is to find all data rows with start_time and end_time
satisfying this expression:

start_time < const_et and end_time >= const_st or end_time is null.
In a FilterList, I can choose either MUST_PASS_ALL or MUST_PASS_ONE, but
none is applicable to this use case.

It would be nice if there is a temporal filter that allows me to select
data valid between [const_st, const_et] (and that end_time is null will be
automatically interpreted as valid up to now).

My domain is not traditionally Internet area, but I am sure folks in
clickstream business have a similar need. And I am wondering how they solve
this problem.

Temporal is commonly supported in traditional databases. So maybe HBase can
offer the same? I guess the current version does not have this support, and
a customer filter needs to be written by myself. I could be wrong. Please
enlighten me.
Shumin Wu

On Mon, Sep 17, 2012 at 8:16 PM, Anoop Sam John <[EMAIL PROTECTED]> wrote:

> Hi
> start_time and end_time are 2 qualifiers in your table.
> You can use a FilterList with MUST_PASS_ALL ( AND condition)
> Add SingleColumnValueFilter for each of the qualifier with the value and
> condition..
>
> -Anoop-
>
> ________________________________________
> From: Shumin Wu [[EMAIL PROTECTED]]
> Sent: Monday, September 17, 2012 9:58 PM
> To: [EMAIL PROTECTED]
> Subject: Temporal in Hbase?
>
> Hi,
>
> I have a user case to "filter" out rows using an "as of" predicate. For
> example, given a specific time point T, I would like to find all rows where
> start_time<=T<=end_time.
>
> Example hbase table schema:
> row_key, col_A, col_B, start_time, end_time
>
> I am wondering if there is any existing filter that allows me to do this.
>
> If not, I guess I would have to write my own custom filter.
>
> Thanks,
>
> Shumin Wu
>
+
Ramkrishna.S.Vasudevan 2012-10-11, 05:32
+
Shumin Wu 2012-10-11, 17:38
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB