Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> datanodes with differenet cpus&mem


+
hua beatls 2013-02-26, 18:02
+
hua beatls 2013-02-26, 18:03
+
Jean-Marc Spaggiari 2013-02-26, 18:08
+
hua beatls 2013-02-26, 18:21
+
Jean-Marc Spaggiari 2013-02-26, 18:30
+
hua beatls 2013-02-27, 01:32
Copy link to this message
-
Re: datanodes with differenet cpus&mem
Hi Hua,

Hadoop will spread the data evenly between all the nodes. And HBase
balancer too.

However, you can still assign more map/reduce jobs to the bigger
servers so they will try to handle more load.

Else, you can manually (or with a customed balancer) move more regions
to the bigger servers.

And even without all those modifications, you can still use all the 8
servers together.

JM

2013/2/26 hua beatls <[EMAIL PROTECTED]>:
> Hi,
>    Yes, i already have these servers, and hope can find solution to use
> them  efficiently.
>    does hadoop/hbase system has the 'intelligence' to arrange more
> work(load, lookup,etc.) to the 8cpu servers compared with 2cpu servers?
>
>
>    thanks!
>
>    beatls
>
>
>
>
> On Wed, Feb 27, 2013 at 2:30 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Hua,
>>
>> It all depend what you want to do with your servers. They are already
>> pretty big. They will be able to handle big load.
>>
>> What's about the hard drives?
>>
>> What can of load are you going to apply? CPU intensive (Like big
>> regular expression on big text content to extract keywors from rows)?
>> Disk intensive (Like fast MRs job storring many data in an output
>> table)? Network intensive (Big transfers, lookups, etc.)?
>>
>> Also, is it for a lab? Or a production?
>>
>> If you already have those servers, you can simply install them with
>> HBase/Hadoop and run some load tests base on your usecase?
>>
>> JM
>>
>> 2013/2/26 hua beatls <[EMAIL PROTECTED]>:
>> > Hi JM,
>> >     should I add more storage to the bigger ones ?
>> >
>> >    many thanks!
>> >
>> >
>> > beatls
>> >
>> > On Wed, Feb 27, 2013 at 2:08 AM, Jean-Marc Spaggiari <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> >> Hi Hua,
>> >>
>> >> You can use servers with different configuration, but the entire
>> >> cluster will tend to run as slow as your slowest server. However, that
>> >> will still help to to spread the load.
>> >>
>> >> JM
>> >>
>> >> 2013/2/26 hua beatls <[EMAIL PROTECTED]>:
>> >> > shoudl we use PC servers with same configuration?
>> >> >
>> >> > beatlsd
>> >> >
>> >> > On Wed, Feb 27, 2013 at 2:02 AM, hua beatls <[EMAIL PROTECTED]> wrote:
>> >> >
>> >> >> HI,
>> >> >>    i have some existing PC servers, which would be used to form a
>> hadoop
>> >> >> cluster, among which 4 servers have 2cpu(6 cores), 96Gmemory, and
>> >> another 4
>> >> >> have 8cpu(6ocres),256G memory. Big gap.
>> >> >>    can i user these 8 servers to make a hadoop cluster? any
>> suggestion?
>> >> >>
>> >> >>
>> >> >>   Thanks!
>> >> >>
>> >> >> beatls
>> >> >>
>> >>
>>