Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> MySQL instance on hadoop name node server in production ?


Copy link to this message
-
Re: MySQL instance on hadoop name node server in production ?
few writes understood but many reads will depends on how big your metastore
will grow.

also what size of hdfs you are looking at that will play a huge factor in
this cause namenode will contain the FSImage in memory so you need a larger
memory given to namenode. Potentially if its a live cluster then I guess
your namenode will be under heavy loads if you are setting up a production
system.

i would still suggest for metastore you setup a couple of VMs with mysql
replication enabled. so the reads can be distributed with load balancers
etc
On Thu, Mar 28, 2013 at 12:00 AM, Sanjay Subramanian <
[EMAIL PROTECTED]> wrote:

>  Thanks Nitin.
> The mysql instance is for hive metastore only so few writes, many reads.
> Regards
> sanjay
>
>   From: Nitin Pawar <[EMAIL PROTECTED]>
> Reply-To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Date: Wednesday, March 27, 2013 11:13 AM
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Subject: Re: MySQL instance on hadoop name node server in production ?
>
>   ideally when you setup namenode and jobtracker in any hadoop cluster on
> production those are normally dedicated to that purpose only.
>
>  Depending on how frequently you are going to run queries and how much
> data the hdfs is going to hold is key factor in deciding this.
>
>
> On Wed, Mar 27, 2013 at 11:32 PM, Sanjay Subramanian <
> [EMAIL PROTECTED]> wrote:
>
>> Hi all
>>
>> I am planning to install mysql server (as hive metastore) on the same box
>> as my name node.
>> My name node has 16GB RAM and hopefully I can get 2TB
>> Any problems with mysql on the dame node as name node ?
>>
>> Thanks
>> sanjay
>>
>>
>> CONFIDENTIALITY NOTICE
>> =====================>> This email message and any attachments are for the exclusive use of the
>> intended recipient(s) and may contain confidential and privileged
>> information. Any unauthorized review, use, disclosure or distribution is
>> prohibited. If you are not the intended recipient, please contact the
>> sender by reply email and destroy all copies of the original message along
>> with any attachments, from your computer system. If you are the intended
>> recipient, please be advised that the content of this message is subject to
>> access, review and disclosure by the sender's Email System Administrator.
>>
>>
>
>
>  --
> Nitin Pawar
>
> CONFIDENTIALITY NOTICE
> =====================> This email message and any attachments are for the exclusive use of the
> intended recipient(s) and may contain confidential and privileged
> information. Any unauthorized review, use, disclosure or distribution is
> prohibited. If you are not the intended recipient, please contact the
> sender by reply email and destroy all copies of the original message along
> with any attachments, from your computer system. If you are the intended
> recipient, please be advised that the content of this message is subject to
> access, review and disclosure by the sender's Email System Administrator.
>

--
Nitin Pawar
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB