|
raviprasad.t@...
2012-10-10, 10:59
Jean-Marc Spaggiari
2012-10-10, 11:52
Anoop Sam John
2012-10-10, 12:45
Doug Meil
2012-10-10, 13:25
raviprasad.t@...
2012-10-10, 14:21
Anoop Sam John
2012-10-11, 04:01
Ramkrishna.S.Vasudevan
2012-10-11, 04:52
yutoo yanio
2012-10-11, 09:37
raviprasad.t@...
2012-10-11, 10:36
Nitin Pawar
2012-10-11, 10:55
|
-
HBase table - distinct valuesraviprasad.t@... 2012-10-10, 10:59
Hi all,
Is it possible to select distinct value from Hbase table. Example :- what is the equivalant code for the below Oracle code in Hbase ? Select count (distinct deptno) from emp ; Regards Raviprasad. T This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com
-
Re: HBase table - distinct valuesJean-Marc Spaggiari 2012-10-10, 11:52
Hi Raviprasad,
What you can do, if deptno is you key, or the first part of you key, is to scan for the first entry, then increment it by one and scan with this value. Let's take an example. Key format is DEPTNO + ID (XXYY) You table content is: 0101 0102 0106 0207 0212 0243 0419 0441 If you scan for the first entry you will find 0101. You extract 01 form that. Then you search for the next DEPTID just above this one. You first key for the search will be 02. And you will find 0207. You do the same. Start key for your search will be 03 and you will find 04. And so on. the main issue you will have is that is you have only few DEPTNO, you will hotspot one server, then another one, and so on. So may you should think about your schema first. Like you can have a table with only the deptno so to get the list of distincts deptno you just scan this table? etc. JM 2012/10/10, [EMAIL PROTECTED] <[EMAIL PROTECTED]>: > Hi all, > Is it possible to select distinct value from Hbase table. > > Example :- > what is the equivalant code for the below Oracle code in Hbase ? > > Select count (distinct deptno) from emp ; > > Regards > Raviprasad. T > > > This e-Mail may contain proprietary and confidential information and is sent > for the intended recipient(s) only. If by an addressing or transmission > error this mail has been misdirected to you, you are requested to delete > this mail immediately. You are also hereby notified that any use, any form > of reproduction, dissemination, copying, disclosure, modification, > distribution and/or publication of this e-mail message, contents or its > attachment other than by its intended recipient/s is strictly prohibited. > > Visit us at http://www.polarisFT.com >
-
RE: HBase table - distinct valuesAnoop Sam John 2012-10-10, 12:45
Hi Your schema? 'deptno' is a cf:qualifier? -Anoop- ________________________________________ From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] Sent: Wednesday, October 10, 2012 4:29 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: HBase table - distinct values Hi all, Is it possible to select distinct value from Hbase table. Example :- what is the equivalant code for the below Oracle code in Hbase ? Select count (distinct deptno) from emp ; Regards Raviprasad. T This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com
-
Re: HBase table - distinct valuesDoug Meil 2012-10-10, 13:25
Typically this is something done as a MapReduce job. http://hbase.apache.org/book.html#mapreduce.example 7.2.4. HBase MapReduce Summary to HBase Example However, if this is an operation to be performed frequently by an application then doing frequent MapReduce jobs for summaries probably isn't the best idea. Either produce periodic summaries into another Hbase table, or denormalize and keep track of the required summaries upon data load. On 10/10/12 6:59 AM, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote: >Hi all, > Is it possible to select distinct value from Hbase table. > >Example :- > what is the equivalant code for the below Oracle code in Hbase ? > > Select count (distinct deptno) from emp ; > >Regards >Raviprasad. T > > >This e-Mail may contain proprietary and confidential information and is >sent for the intended recipient(s) only. If by an addressing or >transmission error this mail has been misdirected to you, you are >requested to delete this mail immediately. You are also hereby notified >that any use, any form of reproduction, dissemination, copying, >disclosure, modification, distribution and/or publication of this e-mail >message, contents or its attachment other than by its intended >recipient/s is strictly prohibited. > >Visit us at http://www.polarisFT.com
-
RE: HBase table - distinct valuesraviprasad.t@... 2012-10-10, 14:21
Hi,
Hbase table name :- emp Column family :- cf Under the column family cf we will be having the field name deptno Regards Raviprasad. T Mobile :- 91- 9894769541 -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> From: Anoop Sam John <[EMAIL PROTECTED]> Date: 10/10/2012 06:18PM Subject: RE: HBase table - distinct values Hi Your schema? 'deptno' is a cf:qualifier? -Anoop- ________________________________________ From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] Sent: Wednesday, October 10, 2012 4:29 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: HBase table - distinct values Hi all, Is it possible to select distinct value from Hbase table. Example :- what is the equivalant code for the below Oracle code in Hbase ? Select count (distinct deptno) from emp ; Regards Raviprasad. T This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com
-
RE: HBase table - distinct valuesAnoop Sam John 2012-10-11, 04:01
Hi Ravi
If dept_no is a CF:qualifier, to know all the dept numbers (distinct or not) you need a full table scan. As Doug said if it is a frequent online query don't think MR is a good choice.. If the data in your emp table is huge a full table scan also wont be that good I feel.... Can you guys think about storing dept number in another table? If you people need query like select <empdetails> from emp where dept_no=? (this query also) you can think about creating secondary index implementation and indexing dept_no... You can use index table for above query as well as the 1st one you asked about. :) -Anoop- ________________________________________ From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] Sent: Wednesday, October 10, 2012 7:51 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: RE: HBase table - distinct values Hi, Hbase table name :- emp Column family :- cf Under the column family cf we will be having the field name deptno Regards Raviprasad. T Mobile :- 91- 9894769541 -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> From: Anoop Sam John <[EMAIL PROTECTED]> Date: 10/10/2012 06:18PM Subject: RE: HBase table - distinct values Hi Your schema? 'deptno' is a cf:qualifier? -Anoop- ________________________________________ From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] Sent: Wednesday, October 10, 2012 4:29 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: HBase table - distinct values Hi all, Is it possible to select distinct value from Hbase table. Example :- what is the equivalant code for the below Oracle code in Hbase ? Select count (distinct deptno) from emp ; Regards Raviprasad. T This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com
-
RE: HBase table - distinct valuesRamkrishna.S.Vasudevan 2012-10-11, 04:52
Are you planning to use region splits? Can the rowkey have the deptno?
Having dept no in another table, may be having a reverse mapping of deptno to empno may be helpful too if such queries are frequent. Regards Ram > -----Original Message----- > From: Anoop Sam John [mailto:[EMAIL PROTECTED]] > Sent: Thursday, October 11, 2012 9:32 AM > To: [EMAIL PROTECTED] > Cc: [EMAIL PROTECTED] > Subject: RE: HBase table - distinct values > > Hi Ravi > If dept_no is a CF:qualifier, to know all the dept numbers > (distinct or not) you need a full table scan. As Doug said if it is a > frequent online query don't think MR is a good choice.. If the data in > your emp table is huge a full table scan also wont be that good I > feel.... Can you guys think about storing dept number in another > table? If you people need query like select <empdetails> from emp > where dept_no=? (this query also) you can think about creating > secondary index implementation and indexing dept_no... You can use > index table for above query as well as the 1st one you asked about. :) > > -Anoop- > ________________________________________ > From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] > Sent: Wednesday, October 10, 2012 7:51 PM > To: [EMAIL PROTECTED] > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: RE: HBase table - distinct values > > Hi, > Hbase table name :- emp > Column family :- cf > Under the column family cf we will be having the field name > deptno > > > Regards > Raviprasad. T > Mobile :- 91- 9894769541 > > > -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "hbase- > [EMAIL PROTECTED]" <[EMAIL PROTECTED]> > From: Anoop Sam John <[EMAIL PROTECTED]> > Date: 10/10/2012 06:18PM > Subject: RE: HBase table - distinct values > > Hi > Your schema? 'deptno' is a cf:qualifier? > > -Anoop- > ________________________________________ > From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] > Sent: Wednesday, October 10, 2012 4:29 PM > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: HBase table - distinct values > > Hi all, > Is it possible to select distinct value from Hbase table. > > Example :- > what is the equivalant code for the below Oracle code in Hbase ? > > Select count (distinct deptno) from emp ; > > Regards > Raviprasad. T > > > This e-Mail may contain proprietary and confidential information and is > sent for the intended recipient(s) only. If by an addressing or > transmission error this mail has been misdirected to you, you are > requested to delete this mail immediately. You are also hereby notified > that any use, any form of reproduction, dissemination, copying, > disclosure, modification, distribution and/or publication of this e- > mail message, contents or its attachment other than by its intended > recipient/s is strictly prohibited. > > Visit us at http://www.polarisFT.com > > > This e-Mail may contain proprietary and confidential information and is > sent for the intended recipient(s) only. If by an addressing or > transmission error this mail has been misdirected to you, you are > requested to delete this mail immediately. You are also hereby notified > that any use, any form of reproduction, dissemination, copying, > disclosure, modification, distribution and/or publication of this e- > mail message, contents or its attachment other than by its intended > recipient/s is strictly prohibited. > > Visit us at http://www.polarisFT.com
-
Re: HBase table - distinct valuesyutoo yanio 2012-10-11, 09:37
you can create a table with key=deptno or every value that you need
distinct value of it. scan of this table shows the distinct values. On Thu, Oct 11, 2012 at 8:22 AM, Ramkrishna.S.Vasudevan < [EMAIL PROTECTED]> wrote: > Are you planning to use region splits? Can the rowkey have the deptno? > > Having dept no in another table, may be having a reverse mapping of deptno > to empno may be helpful too if such queries are frequent. > > Regards > Ram > > -----Original Message----- > > From: Anoop Sam John [mailto:[EMAIL PROTECTED]] > > Sent: Thursday, October 11, 2012 9:32 AM > > To: [EMAIL PROTECTED] > > Cc: [EMAIL PROTECTED] > > Subject: RE: HBase table - distinct values > > > > Hi Ravi > > If dept_no is a CF:qualifier, to know all the dept numbers > > (distinct or not) you need a full table scan. As Doug said if it is a > > frequent online query don't think MR is a good choice.. If the data in > > your emp table is huge a full table scan also wont be that good I > > feel.... Can you guys think about storing dept number in another > > table? If you people need query like select <empdetails> from emp > > where dept_no=? (this query also) you can think about creating > > secondary index implementation and indexing dept_no... You can use > > index table for above query as well as the 1st one you asked about. :) > > > > -Anoop- > > ________________________________________ > > From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] > > Sent: Wednesday, October 10, 2012 7:51 PM > > To: [EMAIL PROTECTED] > > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] > > Subject: RE: HBase table - distinct values > > > > Hi, > > Hbase table name :- emp > > Column family :- cf > > Under the column family cf we will be having the field name > > deptno > > > > > > Regards > > Raviprasad. T > > Mobile :- 91- 9894769541 > > > > > > -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- > > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "hbase- > > [EMAIL PROTECTED]" <[EMAIL PROTECTED]> > > From: Anoop Sam John <[EMAIL PROTECTED]> > > Date: 10/10/2012 06:18PM > > Subject: RE: HBase table - distinct values > > > > Hi > > Your schema? 'deptno' is a cf:qualifier? > > > > -Anoop- > > ________________________________________ > > From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] > > Sent: Wednesday, October 10, 2012 4:29 PM > > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > > Subject: HBase table - distinct values > > > > Hi all, > > Is it possible to select distinct value from Hbase table. > > > > Example :- > > what is the equivalant code for the below Oracle code in Hbase ? > > > > Select count (distinct deptno) from emp ; > > > > Regards > > Raviprasad. T > > > > > > This e-Mail may contain proprietary and confidential information and is > > sent for the intended recipient(s) only. If by an addressing or > > transmission error this mail has been misdirected to you, you are > > requested to delete this mail immediately. You are also hereby notified > > that any use, any form of reproduction, dissemination, copying, > > disclosure, modification, distribution and/or publication of this e- > > mail message, contents or its attachment other than by its intended > > recipient/s is strictly prohibited. > > > > Visit us at http://www.polarisFT.com > > > > > > This e-Mail may contain proprietary and confidential information and is > > sent for the intended recipient(s) only. If by an addressing or > > transmission error this mail has been misdirected to you, you are > > requested to delete this mail immediately. You are also hereby notified > > that any use, any form of reproduction, dissemination, copying, > > disclosure, modification, distribution and/or publication of this e- > > mail message, contents or its attachment other than by its intended > > recipient/s is strictly prohibited. > > > > Visit us at http://www.polarisFT.com>
-
RE: HBase table - distinct valuesraviprasad.t@... 2012-10-11, 10:36
Hi Anoop,
Thanks a lot for your reply, Actually our requirment is just to count the distinct deptno from emp ( Hbase table), We are running various pentaho jobs and we need to test the validity of the results, for that we need the below query. We need a query to select distinct deptno from emp Hbase table. Example :- HBase Table name :- emp, column_family := cf Let us say deptno is the field in the column family cf emp ---- deptno 10 20 30 10 10 10 The Result should be The count (distinct deptno) = 3 We need just the query to know the count of distinct deptno . Thanks Regards Raviprasad. T Mobile :- 91- 9894769541 -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> From: Anoop Sam John <[EMAIL PROTECTED]> Date: 10/11/2012 09:33AM Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> Subject: RE: HBase table - distinct values Hi Ravi If dept_no is a CF:qualifier, to know all the dept numbers (distinct or not) you need a full table scan. As Doug said if it is a frequent online query don't think MR is a good choice.. If the data in your emp table is huge a full table scan also wont be that good I feel.... Can you guys think about storing dept number in another table? If you people need query like select <empdetails> from emp where dept_no=? (this query also) you can think about creating secondary index implementation and indexing dept_no... You can use index table for above query as well as the 1st one you asked about. :) -Anoop- ________________________________________ From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] Sent: Wednesday, October 10, 2012 7:51 PM To: [EMAIL PROTECTED] Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: RE: HBase table - distinct values Hi, Hbase table name :- emp Column family :- cf Under the column family cf we will be having the field name deptno Regards Raviprasad. T Mobile :- 91- 9894769541 -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> From: Anoop Sam John <[EMAIL PROTECTED]> Date: 10/10/2012 06:18PM Subject: RE: HBase table - distinct values Hi Your schema? 'deptno' is a cf:qualifier? -Anoop- ________________________________________ From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] Sent: Wednesday, October 10, 2012 4:29 PM To: [EMAIL PROTECTED]; [EMAIL PROTECTED] Subject: HBase table - distinct values Hi all, Is it possible to select distinct value from Hbase table. Example :- what is the equivalant code for the below Oracle code in Hbase ? Select count (distinct deptno) from emp ; Regards Raviprasad. T This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com This e-Mail may contain proprietary and confidential information and is sent for the intended recipient(s) only. If by an addressing or transmission error this mail has been misdirected to you, you are requested to delete this mail immediately. You are also hereby notified that any use, any form of reproduction, dissemination, copying, disclosure, modification, distribution and/or publication of this e-mail message, contents or its attachment other than by its intended recipient/s is strictly prohibited. Visit us at http://www.polarisFT.com
-
RE: HBase table - distinct valuesNitin Pawar 2012-10-11, 10:55
You may try define a hive table with hbase storage handler n then query it
..though response time will be slow based on how much data you have On Oct 11, 2012 4:19 PM, <[EMAIL PROTECTED]> wrote: > Hi Anoop, > Thanks a lot for your reply, > Actually our requirment is just to count the distinct deptno from emp > ( Hbase table), We are running various pentaho jobs and we need to test > the validity of the results, for that we need the below query. > > We need a query to select distinct deptno from emp Hbase table. > > Example :- > HBase Table name :- emp, column_family := cf > Let us say deptno is the field in the column family cf > > emp > ---- > deptno > 10 > 20 > 30 > 10 > 10 > 10 > > The Result should be > The count (distinct deptno) = 3 > > We need just the query to know the count of distinct deptno . > > Thanks > Regards > Raviprasad. T > Mobile :- 91- 9894769541 > > > -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> > From: Anoop Sam John <[EMAIL PROTECTED]> > Date: 10/11/2012 09:33AM > Cc: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> > Subject: RE: HBase table - distinct values > > Hi Ravi > If dept_no is a CF:qualifier, to know all the dept numbers > (distinct or not) you need a full table scan. As Doug said if it is a > frequent online query don't think MR is a good choice.. If the data in > your emp table is huge a full table scan also wont be that good I feel.... > Can you guys think about storing dept number in another table? If you > people need query like select <empdetails> from emp where dept_no=? (this > query also) you can think about creating secondary index implementation > and indexing dept_no... You can use index table for above query as well as > the 1st one you asked about. :) > > -Anoop- > ________________________________________ > From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] > Sent: Wednesday, October 10, 2012 7:51 PM > To: [EMAIL PROTECTED] > Cc: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: RE: HBase table - distinct values > > Hi, > Hbase table name :- emp > Column family :- cf > Under the column family cf we will be having the field name deptno > > > Regards > Raviprasad. T > Mobile :- 91- 9894769541 > > > -----Anoop Sam John <[EMAIL PROTECTED]> wrote: ----- > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>, " > [EMAIL PROTECTED]" <[EMAIL PROTECTED]> > From: Anoop Sam John <[EMAIL PROTECTED]> > Date: 10/10/2012 06:18PM > Subject: RE: HBase table - distinct values > > Hi > Your schema? 'deptno' is a cf:qualifier? > > -Anoop- > ________________________________________ > From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] > Sent: Wednesday, October 10, 2012 4:29 PM > To: [EMAIL PROTECTED]; [EMAIL PROTECTED] > Subject: HBase table - distinct values > > Hi all, > Is it possible to select distinct value from Hbase table. > > Example :- > what is the equivalant code for the below Oracle code in Hbase ? > > Select count (distinct deptno) from emp ; > > Regards > Raviprasad. T > > > This e-Mail may contain proprietary and confidential information and is > sent for the intended recipient(s) only. If by an addressing or > transmission error this mail has been misdirected to you, you are requested > to delete this mail immediately. You are also hereby notified that any use, > any form of reproduction, dissemination, copying, disclosure, modification, > distribution and/or publication of this e-mail message, contents or its > attachment other than by its intended recipient/s is strictly prohibited. > > Visit us at http://www.polarisFT.com > > > This e-Mail may contain proprietary and confidential information and is > sent for the intended recipient(s) only. If by an addressing or > transmission error this mail has been misdirected to you, you are requested > to delete this mail immediately. You are also hereby notified that any use, |