Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> 0.92 and Read/writes not scaling


Copy link to this message
-
Re: 0.92 and Read/writes not scaling
On Fri, Apr 13, 2012 at 9:06 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Fri, Apr 13, 2012 at 8:02 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>> If you want to patch on the HBase side, you can edit HLog.java to
>> remove the checks for the "sync" method, and have it only call
>> "hflush". It's only the compatibility path that caused the problem.
>>
>
> You mean change the order here boss?

Yep - invoking hflush instead of syncfs should fix the issue on older
0.23.x/CDH4 builds, I think (though I didn't test it). Going forward
it won't matter though.

FYI I verified that the fix made it into our nightly CDH4 build last
night (0.23.1+360)

-Todd

>
>
>  @Override
>  public void sync() throws IOException {
>    if (this.syncFs != null) {
>      try {
>       this.syncFs.invoke(this.writer, HLog.NO_ARGS);
>      } catch (Exception e) {
>        throw new IOException("Reflection", e);
>      }
>    } else if (this.hflush != null) {
>      try {
>        this.hflush.invoke(getWriterFSDataOutputStream(), HLog.NO_ARGS);
>      } catch (Exception e) {
>        throw new IOException("Reflection", e);
>      }
>    }
>  }
>
>
> Call hflush if its available ahead of syncFs?
>
> Seems like we should get this in all around.  I can do it.
>
> Good stuff,
> St.Ack

--
Todd Lipcon
Software Engineer, Cloudera
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB