Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> openTSDB   lose large amount of data when the client are writing


Copy link to this message
-
Re: openTSDB lose large amount of data when the client are writing
On Thu, Sep 19, 2013 at 10:09 AM, Tianying Chang <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I have a customer who use openTSDB. Recently we found that only less than
> 10% data are written, rest are are lost. By checking the RS log, there are
> many row lock related issues, like below. It seems large amount of write to
> tsdb that need row lock caused the problem. Anyone else see similar
> problem?  Is it a bug of openTSDB? Or it is due to HBase exposed a
> vulnerable API?
>
> org.apache.hadoop.hbase.UnknownRowLockException: Invalid row lock
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getLockFromId(HRegionServer.java:2732)
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:2071)
>         at sun.reflect.GeneratedMethodAccessor28.invoke(Unknown Source)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
>         at
> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1426)
> 13/09/18 12:08:30 ERROR regionserver.HRegionServer:
> org.apache.hadoop.hbase.UnknownRowLockException: -6180307918863136448
>         at
> org.apache.hadoop.hbase.regionserver.HRegionServer.unlockRow(HRegionServer.java:2765)
>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>         at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>         at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>         at java.lang.reflect.Method.invoke(Method.java:616)
>         at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:320)
>
>
> Thanks
> Tian-Ying
>
Local filesystem or hdfs?
St.Ack
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB