Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - Active-Active Performance


Copy link to this message
-
Re: Active-Active Performance
Hemanth Yamijala 2010-05-25, 02:19
Anthony,

> In regards to different locations of the HADOOP home this is low
> priority more for testing not production. I was trying to install HADOOP
> for testing over 2 machines with only a Windows XP machine running
> Cygwin and a Mac running Darwin. Not a priority.
>
> In regards to my last question about operating in a detached fashion, we
> are trying to factor in what happens when the link between both sites is
> cut. Will both sites operate independently until the connection is
> re-established? Is there any particular setup required to ensure we can
> cover this scenario or is it an out-of-the-box feature?

When you say 'sites', do you mean two different Hadoop installations ?
In general, each site is independent. So, I am unable to understand
where the link comes in.
>
> Anthony
>
>
> -----Original Message-----
> From: Hemanth Yamijala [mailto:[EMAIL PROTECTED]]
> Sent: Tuesday, 25 May 2010 12:08 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Active-Active Performance
>
> Anthony,
>
> I'm new to Hadoop and I've been given the task to see how we might
> utilise
>> Hadoop and HBase to implement an Active-Active site layer for sharing
>> information across a distributed application.
>>
>>
>>
>> I've been able to:
>>
>> *         Install and get Hadoop running on a single node and am in
> the
>> process of configure a 2 node setup.
>>
>> *         Install HBase on a single node and create a table and
> mapping as
>> well as insert data into the system
>>
>>
>>
>> Once I've got the mutli-node configured I hope to run some tests as
> well.
>>
>>
>>
>> I've noticed that trying to start Hadoop in distributed mode, the
> slave
>> will ssh to the master to start it as well (bin/start-all.sh) provided
> the
>> same path is setup on the remote machine.
>>
>>
>>
>> Questions:
>>
>> Can I configure the system IF the Hadoop installation is not in the
> same
>> location per machine?
>>
>
> I would think configuring and managing such a system would get very
> complex
> - for e.g. if you'll want to add nodes to expand in future. You would
> also
> not be able to take advantage of the very helpful scripts that come with
> Hadoop. Is there a reason why you want to do this ?
>
>> If the master node goes down (say due to electrical fault or system
> fault)
>> how do the slave nodes react? Will they continue to run? Will the
> nodes be
>> back in sync once the master starts again?
>>
>
> Hadoop slaves will continue. They will enter a retry loop trying to
> connect
> to the master until it comes up. In doing so, they could fill up log
> files
> very fast though. If the master starts with the same configuration,
> (same
> host, ports), they should be able to connect and resume.
>
>> Would I require a particular configuration to ensure that both our
> sites
>> can operate within the cluster as well as in a detached fashion (due
> to
>> maintenance or network issues)?
>>
>>
>>
> I did not quite follow this. Can you explain a little more about how you
> want to setup your system ?
>
> Thanks
> Hemanth
>
> _____________________________________________________________________
> This e-mail has been scanned for viruses by MCI's Internet Managed
> Scanning Services - powered by MessageLabs. For further information
> visit http://www.mci.com
>
> **********************************************************************
> This e-mail message and any attachments are intended only for the use of the addressee(s) named above and may contain information that is privileged and confidential. If you are not the intended recipient, any display, dissemination, distribution, or copying is strictly prohibited.   If you believe you have received this e-mail message in error, please immediately notify the sender by replying to this e-mail message or by telephone to (02) 9646 9222. Please delete the email and any attachments and do not retain the email or any attachments in any form.
> **********************************************************************
>