Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Re: 0.92 and Read/writes not scaling


+
Juhani Connolly 2012-04-03, 02:52
+
Jonathan Hsieh 2012-04-03, 00:21
+
Juhani Connolly 2012-04-03, 03:02
+
Jonathan Hsieh 2012-04-03, 03:19
+
Stack 2012-04-03, 03:50
+
Jonathan Hsieh 2012-04-03, 16:56
+
Stack 2012-04-03, 17:42
+
Juhani Connolly 2012-04-05, 01:45
+
Ted Yu 2012-04-05, 02:43
+
Juhani Connolly 2012-04-05, 03:02
+
Todd Lipcon 2012-04-14, 03:02
+
Stack 2012-04-14, 04:06
+
Jonathan Hsieh 2012-04-14, 12:43
Copy link to this message
-
Re: 0.92 and Read/writes not scaling
On Fri, Apr 13, 2012 at 9:06 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Fri, Apr 13, 2012 at 8:02 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>> If you want to patch on the HBase side, you can edit HLog.java to
>> remove the checks for the "sync" method, and have it only call
>> "hflush". It's only the compatibility path that caused the problem.
>>
>
> You mean change the order here boss?

Yep - invoking hflush instead of syncfs should fix the issue on older
0.23.x/CDH4 builds, I think (though I didn't test it). Going forward
it won't matter though.

FYI I verified that the fix made it into our nightly CDH4 build last
night (0.23.1+360)

-Todd

>
>
>  @Override
>  public void sync() throws IOException {
>    if (this.syncFs != null) {
>      try {
>       this.syncFs.invoke(this.writer, HLog.NO_ARGS);
>      } catch (Exception e) {
>        throw new IOException("Reflection", e);
>      }
>    } else if (this.hflush != null) {
>      try {
>        this.hflush.invoke(getWriterFSDataOutputStream(), HLog.NO_ARGS);
>      } catch (Exception e) {
>        throw new IOException("Reflection", e);
>      }
>    }
>  }
>
>
> Call hflush if its available ahead of syncFs?
>
> Seems like we should get this in all around.  I can do it.
>
> Good stuff,
> St.Ack

--
Todd Lipcon
Software Engineer, Cloudera
+
Alok Singh 2012-04-02, 17:15
+
Juhani Connolly 2012-04-03, 02:56
+
Stack 2012-04-02, 17:41
+
Juhani Connolly 2012-03-19, 10:41
+
Ramkrishna.S.Vasudevan 2012-03-19, 11:03
+
Mingjian Deng 2012-03-19, 11:56
+
Juhani Connolly 2012-03-19, 12:02
+
Ramkrishna.S.Vasudevan 2012-03-19, 12:27
+
Christian Schäfer 2012-03-19, 12:21
+
Juhani Connolly 2012-03-19, 12:31
+
Matt Corgan 2012-03-19, 16:55
+
Juhani Connolly 2012-03-21, 03:09
+
Mikael Sitruk 2012-03-21, 05:29
+
Juhani Connolly 2012-03-23, 07:40
+
Stack 2012-03-26, 16:37
+
Juhani Connolly 2012-03-23, 07:48
+
Matt Corgan 2012-03-26, 13:58
+
Stack 2012-03-26, 16:42
+
Juhani Connolly 2012-03-26, 17:08
+
Todd Lipcon 2012-03-27, 01:43
+
Juhani Connolly 2012-03-27, 03:18
+
Juhani Connolly 2012-03-28, 08:27
+
Buckley,Ron 2012-03-28, 12:41
+
Stack 2012-03-28, 17:12
+
Buckley,Ron 2012-03-28, 17:56
+
Juhani Connolly 2012-03-29, 02:36
+
Juhani Connolly 2012-03-26, 16:48
+
Mikael Sitruk 2012-03-26, 14:21
+
Juhani Connolly 2012-03-26, 16:59
+
Stack 2012-03-20, 19:27
+
Juhani Connolly 2012-03-19, 11:09
+
Stack 2012-03-26, 16:29
+
Juhani Connolly 2012-03-26, 17:02
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB