Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Parallel scan in HBase


+
Farrokh Shahriari 2013-02-01, 13:52
+
lars hofhansl 2013-02-01, 15:46
+
Mohammad Tariq 2013-02-01, 16:40
+
James Taylor 2013-02-01, 17:07
+
Farrokh Shahriari 2013-02-02, 04:54
+
Mohammad Tariq 2013-02-01, 14:02
+
Farrokh Shahriari 2013-02-01, 14:57
+
Alexander Ignatov 2013-02-01, 15:07
Copy link to this message
-
Re: Parallel scan in HBase
MR job is almost doing that.

The map methode is called for each row, and you can have multiple jobs
running at the same time.

It's the way the rowcounter is working. Scanning every row to count
it, but spreading the work over all the nodes...

Give it a look.

JM

2013/2/1, Alexander Ignatov <[EMAIL PROTECTED]>:
> You could use Coprocessors framework. To do that you have to implement
> your own Coprocessors's module and include it to each RegionServers.
>
> Here is an introduction article how to use Coprocessors:
> https://blogs.apache.org/hbase/entry/coprocessor_introduction
>
> --
> Regards,
> Alexander Ignatov
>
>
> On 2/1/2013 6:57 PM, Farrokh Shahriari wrote:
>> Tnx for your reply,
>> In my case, I should scan all rows( about 1 millions to 5 millions rows)
>> in
>> a table & it takes a long time. I wanna know is there any way I can do it
>> in parallel or not ?
>>
>> On Fri, Feb 1, 2013 at 5:32 PM, Mohammad Tariq <[EMAIL PROTECTED]>
>> wrote:
>>
>>> Hello Farrokh,
>>>
>>>      Scans work sequentially with one region after the other. Scans from
>>> client side do not go to regionservers in parallel. And, for the second
>>> question, the code will run at the client side.
>>>
>>> Warm Regards,
>>> Tariq
>>> https://mtariq.jux.com/
>>> cloudfront.blogspot.com
>>>
>>>
>>> On Fri, Feb 1, 2013 at 7:22 PM, Farrokh Shahriari <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Hi there
>>>> I have two question about scan in Hbase :
>>>> 1) Does scan operation with specific filter run in parallel on
>>>> different
>>>> regionservers ?
>>>> 2) I wanna know whether this code runs at client side for searching the
>>>> retrieved results or not ?
>>>>
>>>>           for (Result result : scanner1) {
>>>>                           for (KeyValue kv : result.raw()) {
>>>>                           //
>>>>                           // some coeds
>>>>                           //
>>>>                       }
>>>>                 }
>>>>
>>>>
>>>> Farrokh Shahriari
>>>>
>
>
>