Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> AggregateProtocol Help


+
Tom Wilcox 2011-12-22, 16:09
+
Ted Yu 2011-12-22, 17:03
+
Tom Wilcox 2011-12-23, 11:02
+
Ted Yu 2011-12-23, 15:04
+
Gary Helmling 2011-12-23, 18:05
+
Royston Sellman 2012-01-01, 17:35
+
Ted Yu 2012-01-01, 17:58
+
Royston Sellman 2012-01-01, 19:26
Copy link to this message
-
Re: AggregateProtocol Help
I think we should try to make AggregateProtocol work first.

I am not with computer at this moment. Will look at the code when I get back.

The table below isn't too big. Experimenting with a smaller table would be nice.

Cheers

On Jan 1, 2012, at 11:26 AM, Royston Sellman <[EMAIL PROTECTED]> wrote:

> Hi Ted,
>
> I think 0 is the only value we ever see (I'll check tomorrow: the server is down right now). Our table has 557,000 rows. I'll try a much shorter table tomorrow.
>
> Yes, we have RS running on the NN, but it's a test cluster and we are used to it :)
>
> Do you think using AggregationProtocol is the best strategy for the case where we want to use basic SQL-style functions like SUM, AVG, STD, MIN, MAX? Do you think there is a better strategy?
>
> Many thanks,
> Royston
>
>
> On 1 Jan 2012, at 17:58, Ted Yu wrote:
>
>> Royston:
>> Happy New Year to you too.
>>
>>>> java.net.SocketTimeoutException: Call to namenode/10.0.0.235:60020failed on
>>
>> It seems the namenode above actually refers to a region server. This is a
>> little bit confusing :-)
>>
>> The sum value below is 0.
>> Have you ever seen a value greater than 0 ?
>>
>> How many rows are there in this CF:CQ ?
>> The timeout was reported earlier by other people where there're many rows
>> in the table.
>>
>> There is a JIRA to provide streaming support for coprocessor but the
>> development there has stalled.
>>
>> Cheers
>>
>> On Sun, Jan 1, 2012 at 9:35 AM, Royston Sellman <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi Gary and Ted,
>>>
>>> Royston (Tom's colleague) here. Back onto this after the Christmas/New Year
>>> break.
>>>
>>> Many thanks for your help so far. We enabled our database via your
>>> hbase-site.xml mod and were able to move on. to other errors. But I think
>>> we
>>> are now actually getting an aggregation partially calculated on our table
>>> (this feels like progress). The details:
>>>
>>> On running our client we now get this exception:
>>> 11/12/31 17:51:09 WARN
>>> client.HConnectionManager$HConnectionImplementation: Error executing for
>>> row
>>>
>>> java.util.concurrent.ExecutionException:
>>> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
>>> attempts=10, exceptions:
>>> Sat Dec 31 17:41:30 GMT 2011,
>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1fc4f0f8,
>>> java.net.SocketTimeoutException: Call to namenode/10.0.0.235:60020 failed
>>> on
>>> socket timeout exception: java.net.SocketTimeoutException: 60000 millis
>>> timeout while waiting for channel to be ready for read. ch :
>>> java.nio.channels.SocketChannel[connected local=/10.0.0.235:59999
>>> remote=namenode/10.0.0.235:60020]
>>> (8 more of these, making for 10 tries)
>>> Sat Dec 31 17:51:09 GMT 2011,
>>> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@1fc4f0f8,
>>> java.net.SocketTimeoutException: Call to namenode/10.0.0.235:60020 failed
>>> on
>>> socket timeout exception: java.net.SocketTimeoutException: 60000 millis
>>> timeout while waiting for channel to be ready for read. ch :
>>> java.nio.channels.SocketChannel[connected local=/10.0.0.235:59364
>>> remote=namenode/10.0.0.235:60020]
>>>
>>>      at
>>> java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:222)
>>>      at java.util.concurrent.FutureTask.get(FutureTask.java:83)
>>>      at
>>>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.
>>> processExecs(HConnectionManager.java:1465)
>>>      at
>>> org.apache.hadoop.hbase.client.HTable.coprocessorExec(HTable.java:1555)
>>>      at
>>>
>>> org.apache.hadoop.hbase.client.coprocessor.AggregationClient.sum(Aggregation
>>> Client.java:229)
>>>      at EDRPAggregator.testSumWithValidRange(EDRPAggregator.java:51)
>>>      at EDRPAggregator.main(EDRPAggregator.java:77)
>>>      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>>      at
>>>
>>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39
>>> )
>>>      at
>>>
>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl
+
Himanshu Vashishtha 2012-01-02, 01:18
+
Ted Yu 2012-01-02, 01:53
+
Gary Helmling 2012-01-02, 06:23
+
Royston Sellman 2012-01-03, 16:32
+
Ted Yu 2012-01-03, 17:09
+
Royston Sellman 2012-01-03, 17:48
+
Ted Yu 2012-01-03, 18:00
+
Himanshu Vashishtha 2012-01-03, 18:14
+
Royston Sellman 2012-01-03, 18:42
+
Ted Yu 2012-01-03, 18:58
+
Ted Yu 2012-01-03, 21:31
+
Royston Sellman 2012-01-04, 11:43
+
Ted Yu 2012-01-04, 15:01
+
Royston Sellman 2012-01-04, 18:57
+
Himanshu Vashishtha 2012-01-03, 17:11
+
Royston Sellman 2012-01-03, 17:54
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB