Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Regionserver goes down while endpoint execution


Copy link to this message
-
Re: Regionserver goes down while endpoint execution
Himanshu Vashishtha 2013-03-12, 16:59
I don't see RS dying with this. It says that it is taking more time
than 60 sec (default timeout for clients), and therefore it stops
processing the coprocessor call as the client is disconnected.
Is your cluster okay? how many rows in the table? Normal scan works
good? Can you share more about your cluster details (nodes, tables,
regions, data size, etc)?

Thanks,
Himanshu

On Tue, Mar 12, 2013 at 4:46 AM, Kumar, Deepak8 <[EMAIL PROTECTED]> wrote:
> Lars,
>
> I am getting following errors at datanode & region servers.
>
>
>
> Regards,
>
> Deepak
>
>
>
> From: Kumar, Deepak8 [CCC-OT_IT NE]
> Sent: Tuesday, March 12, 2013 3:00 AM
> To: Kumar, Deepak8 [CCC-OT_IT NE]; '[EMAIL PROTECTED]'; 'lars hofhansl'
>
>
> Subject: RE: Regionserver goes down while endpoint execution
>
>
>
> Lars,
>
> It is having following errors when I execute the Endpoint RPC client from
> eclipse. It seems some of the regions at regionserver
> vm-8aa9-fe74.nam.nsroot.net is taking more time to reponse.
>
>
>
> Could you guide how to fix it. I don’t find any option to set
> hbase.rpc.timeout from hbase configuration menu in CDH4 CM server for hbase
> configuration.
>
>
>
> Regards,
>
> Deepak
>
>
>
> 3/03/12 02:33:12 INFO zookeeper.ClientCnxn: Session establishment complete
> on server vm-15c2-3bbf.nam.nsroot.net/10.96.172.44:2181, sessionid > 0x53d591b77090026, negotiated timeout = 60000
>
> Mar 12, 2013 2:33:13 AM org.apache.hadoop.conf.Configuration
> warnOnceIfDeprecated
>
> WARNING: hadoop.native.lib is deprecated. Instead, use
> io.native.lib.available
>
> Mar 12, 2013 2:44:00 AM
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation
> processExecs
>
> WARNING: Error executing for row
> 153299:1362780381523:2932572079500658:vm-ab1f-dd21.nam.nsroot.net:
>
> java.util.concurrent.ExecutionException:
> org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after
> attempts=10, exceptions:
>
> Tue Mar 12 02:34:15 EDT 2013,
> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@39443f,
> java.net.SocketTimeoutException: Call to
> vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020 failed on socket timeout
> exception: java.net.SocketTimeoutException: 60000 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/150.110.96.212:2271
> remote=vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020]
>
> Tue Mar 12 02:35:16 EDT 2013,
> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@39443f,
> java.net.SocketTimeoutException: Call to
> vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020 failed on socket timeout
> exception: java.net.SocketTimeoutException: 60000 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/150.110.96.212:2403
> remote=vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020]
>
> Tue Mar 12 02:36:18 EDT 2013,
> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@39443f,
> java.net.SocketTimeoutException: Call to
> vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020 failed on socket timeout
> exception: java.net.SocketTimeoutException: 60000 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/150.110.96.212:2465
> remote=vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020]
>
> Tue Mar 12 02:37:20 EDT 2013,
> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@39443f,
> java.net.SocketTimeoutException: Call to
> vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020 failed on socket timeout
> exception: java.net.SocketTimeoutException: 60000 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/150.110.96.212:2500
> remote=vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020]
>
> Tue Mar 12 02:38:22 EDT 2013,
> org.apache.hadoop.hbase.ipc.ExecRPCInvoker$1@39443f,
> java.net.SocketTimeoutException: Call to
> vm-8aa9-fe74.nam.nsroot.net/10.42.105.91:60020 failed on socket timeout
> exception: java.net.SocketTimeoutException: 60000 millis timeout while