I want 20 servers, I got 7, so I want to make the most of the 7 I have. Each
of the 7 servers have: 24GB of ram, 4TB, and 8 cores.

 

Would it be terribly unwise of me to Run such a configuration:

 

.         Server #1:           NameNode + Master + TaskTracker(reduced
slots)

.         Server #2:           CheckpointNode(aka Secondary Name Node) +
TaskTracker(slightly reduced slots)

.         Server #3:           TaskTracker

.         Server #4:           TaskTracker

.         Server #5:           TaskTracker

.         Server #6:           TaskTracker

.         Server #7:           TaskTracker

 

Did I miss anything? Did I shoot myself in the foot anywhere?

 

 

I think its fine to keep it so long as you also monitor the nodes for
issues during heavy load times and tweak slave configs appropriately
when some resource is overused.

On Mon, Mar 18, 2013 at 3:54 PM, David Parks <[EMAIL PROTECTED]> wrote:
> I want 20 servers, I got 7, so I want to make the most of the 7 I have. Each
> of the 7 servers have: 24GB of ram, 4TB, and 8 cores.
>
>
>
> Would it be terribly unwise of me to Run such a configuration:
>
>
>
> ·         Server #1:           NameNode + Master + TaskTracker(reduced
> slots)
>
> ·         Server #2:           CheckpointNode(aka Secondary Name Node) +
> TaskTracker(slightly reduced slots)
>
> ·         Server #3:           TaskTracker
>
> ·         Server #4:           TaskTracker
>
> ·         Server #5:           TaskTracker
>
> ·         Server #6:           TaskTracker
>
> ·         Server #7:           TaskTracker
>
>
>
> Did I miss anything? Did I shoot myself in the foot anywhere?
>
>
>
>

--
Harsh J

David,

You didn't look at how resource intensive your map/reduce jobs are.

Best regards,

Jens

Good points all,

 

The mapreduce jobs are, well. intensive. We've got a whole variety, but
typically I see them use a lot of CPU, a lot of Disk, and upon occasion a
whole bunch of Network bandwidth.  Duh right?  J

 

The master node is mostly CPU intensive right? We're using LXC to segregate
(psudo-virtualize) our environments for ease of development and management.
I'm looking into whether I can use LXC's quota system to guarantee a certain
level of CPU resources to the container where the master node is housed. If
I can do that I guess we wouldn't have any issue here.

 

Thanks!

David

 

 

From: Jens Scheidtmann [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, March 19, 2013 3:12 PM
To: [EMAIL PROTECTED]
Subject: Re: On a small cluster can we double up namenode/master with
tasktrackers?

 

David,

You didn't look at how resource intensive your map/reduce jobs are.

 

Best regards,

Jens