Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - Re: 0.92 and Read/writes not scaling


+
Juhani Connolly 2012-04-03, 02:52
+
Jonathan Hsieh 2012-04-03, 00:21
+
Juhani Connolly 2012-04-03, 03:02
+
Jonathan Hsieh 2012-04-03, 03:19
+
Stack 2012-04-03, 03:50
+
Jonathan Hsieh 2012-04-03, 16:56
+
Stack 2012-04-03, 17:42
+
Juhani Connolly 2012-04-05, 01:45
+
Ted Yu 2012-04-05, 02:43
+
Juhani Connolly 2012-04-05, 03:02
+
Todd Lipcon 2012-04-14, 03:02
+
Stack 2012-04-14, 04:06
+
Jonathan Hsieh 2012-04-14, 12:43
Copy link to this message
-
Re: 0.92 and Read/writes not scaling
Todd Lipcon 2012-04-15, 06:28
On Fri, Apr 13, 2012 at 9:06 PM, Stack <[EMAIL PROTECTED]> wrote:
> On Fri, Apr 13, 2012 at 8:02 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
>> If you want to patch on the HBase side, you can edit HLog.java to
>> remove the checks for the "sync" method, and have it only call
>> "hflush". It's only the compatibility path that caused the problem.
>>
>
> You mean change the order here boss?

Yep - invoking hflush instead of syncfs should fix the issue on older
0.23.x/CDH4 builds, I think (though I didn't test it). Going forward
it won't matter though.

FYI I verified that the fix made it into our nightly CDH4 build last
night (0.23.1+360)

-Todd

>
>
>  @Override
>  public void sync() throws IOException {
>    if (this.syncFs != null) {
>      try {
>       this.syncFs.invoke(this.writer, HLog.NO_ARGS);
>      } catch (Exception e) {
>        throw new IOException("Reflection", e);
>      }
>    } else if (this.hflush != null) {
>      try {
>        this.hflush.invoke(getWriterFSDataOutputStream(), HLog.NO_ARGS);
>      } catch (Exception e) {
>        throw new IOException("Reflection", e);
>      }
>    }
>  }
>
>
> Call hflush if its available ahead of syncFs?
>
> Seems like we should get this in all around.  I can do it.
>
> Good stuff,
> St.Ack

--
Todd Lipcon
Software Engineer, Cloudera
+
Alok Singh 2012-04-02, 17:15
+
Juhani Connolly 2012-04-03, 02:56
+
Stack 2012-04-02, 17:41
+
Juhani Connolly 2012-03-19, 10:41
+
Ramkrishna.S.Vasudevan 2012-03-19, 11:03
+
Mingjian Deng 2012-03-19, 11:56
+
Juhani Connolly 2012-03-19, 12:02
+
Ramkrishna.S.Vasudevan 2012-03-19, 12:27
+
Christian Schäfer 2012-03-19, 12:21
+
Juhani Connolly 2012-03-19, 12:31
+
Matt Corgan 2012-03-19, 16:55
+
Juhani Connolly 2012-03-21, 03:09
+
Mikael Sitruk 2012-03-21, 05:29
+
Juhani Connolly 2012-03-23, 07:40
+
Stack 2012-03-26, 16:37
+
Juhani Connolly 2012-03-23, 07:48
+
Matt Corgan 2012-03-26, 13:58
+
Stack 2012-03-26, 16:42
+
Juhani Connolly 2012-03-26, 17:08
+
Todd Lipcon 2012-03-27, 01:43
+
Juhani Connolly 2012-03-27, 03:18
+
Juhani Connolly 2012-03-28, 08:27
+
Buckley,Ron 2012-03-28, 12:41
+
Stack 2012-03-28, 17:12
+
Buckley,Ron 2012-03-28, 17:56
+
Juhani Connolly 2012-03-29, 02:36
+
Juhani Connolly 2012-03-26, 16:48
+
Mikael Sitruk 2012-03-26, 14:21
+
Juhani Connolly 2012-03-26, 16:59
+
Stack 2012-03-20, 19:27
+
Juhani Connolly 2012-03-19, 11:09
+
Stack 2012-03-26, 16:29
+
Juhani Connolly 2012-03-26, 17:02