Varun Sharma 2012-12-24, 17:27
Stack 2012-12-26, 21:13
Varun Sharma 2012-12-28, 08:51
-Re: Where to place logs
Leonid Fedotov 2012-12-28, 17:22
this really depends on your log rotation and retention policy.
Logs usually pretty big, but if you rotate it once a day(for example) and remove old logs after, say, 1 week, you probably will not need to have huge amount of space for it…
You should estimate logs side before making decision.
On Dec 28, 2012, at 12:51 AM, Varun Sharma wrote:
> Thanks for the tip. I was just wondering what people have done in the past
> - do people typically reserve a separate disk for logging activity ?
> On Wed, Dec 26, 2012 at 1:13 PM, Stack <[EMAIL PROTECTED]> wrote:
>> On Mon, Dec 24, 2012 at 9:27 AM, Varun Sharma <[EMAIL PROTECTED]> wrote:
>>> I am wondering where people usually place hbase + hadoop logs. I have 4
>>> disks and 1 very tiny disk with barely 500 megs (thats the typical setup
>>> amazon ec2). The 4 disks shall be used for hbase data. Since 500M is too
>>> small, should I place logs on one of the 4 disks. Could it potentially
>>> steal IOP(s) from hbase ? Does anyone have an idea how much of an
>>> logging really is ?
>> Looks like you have no choice but to put your logging on a data disk.
>> Logging can be expensive, yes. It should be easy enough to measure in
>> your setup. You can disable all logging for a short period while the
>> cluster is under load; do it on a single node even (you can do this from
>> the UI). See how your io changes when logging is off.