Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> MySQL instance on hadoop name node server in production ?


Copy link to this message
-
Re: MySQL instance on hadoop name node server in production ?
Thanks Nitin…the mysql replication is a good point
Regards
sanjay

From: Nitin Pawar <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Wednesday, March 27, 2013 11:36 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: MySQL instance on hadoop name node server in production ?

few writes understood but many reads will depends on how big your metastore will grow.

also what size of hdfs you are looking at that will play a huge factor in this cause namenode will contain the FSImage in memory so you need a larger memory given to namenode. Potentially if its a live cluster then I guess your namenode will be under heavy loads if you are setting up a production system.

i would still suggest for metastore you setup a couple of VMs with mysql replication enabled. so the reads can be distributed with load balancers etc
On Thu, Mar 28, 2013 at 12:00 AM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Thanks Nitin.
The mysql instance is for hive metastore only so few writes, many reads.
Regards
sanjay

From: Nitin Pawar <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Wednesday, March 27, 2013 11:13 AM
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: MySQL instance on hadoop name node server in production ?

ideally when you setup namenode and jobtracker in any hadoop cluster on production those are normally dedicated to that purpose only.

Depending on how frequently you are going to run queries and how much data the hdfs is going to hold is key factor in deciding this.
On Wed, Mar 27, 2013 at 11:32 PM, Sanjay Subramanian <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi all

I am planning to install mysql server (as hive metastore) on the same box
as my name node.
My name node has 16GB RAM and hopefully I can get 2TB
Any problems with mysql on the dame node as name node ?

Thanks
sanjay
CONFIDENTIALITY NOTICE
=====================This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.
--
Nitin Pawar

CONFIDENTIALITY NOTICE
=====================This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.

--
Nitin Pawar

CONFIDENTIALITY NOTICE
=====================This email message and any attachments are for the exclusive use of the intended recipient(s) and may contain confidential and privileged information. Any unauthorized review, use, disclosure or distribution is prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message along with any attachments, from your computer system. If you are the intended recipient, please be advised that the content of this message is subject to access, review and disclosure by the sender's Email System Administrator.