Hi,
Im trying to get Duo core computer to run 2 data nodes on Hadoop.
What do I have to do to add the other node as a slave?
--
View this message in context: http://old.nabble.com/How-to-run-two-data-nodes-on-one-pc--tp33763649p33763649.html
Sent from the HBase User mailing list archive at Nabble.com.

Which is your current hardware configuration?
1- One way is to use two VMs in Vmware Workstation, Vmware Server, or
Virtualbox, with similar configuration for these two DNs, but
you should have in mind that HBase needs a lot of RAM to work properly,
so I don't recommend to use a single machine to run two DN.
On 05/09/2012 02:17 PM, MapHadoop wrote:
> Hi,
> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
> What do I have to do to add the other node as a slave?

--
Marcos Luis Ort�z Valmaseda (@marcosluis2186)
  Data Engineer at UCI
  http://marcosluis2186.posterous.com

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci

Too small a machine.

Better question. Why do you want to get 2 nodes on one machine?

On May 9, 2012, at 1:49 PM, Marcos Ortiz wrote:

> Which is your current hardware configuration?
> 1- One way is to use two VMs in Vmware Workstation, Vmware Server, or Virtualbox, with similar configuration for these two DNs, but
> you should have in mind that HBase needs a lot of RAM to work properly, so I don't recommend to use a single machine to run two DN.
>
>
> On 05/09/2012 02:17 PM, MapHadoop wrote:
>> Hi,
>> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
>> What do I have to do to add the other node as a slave?
>
> --
> Marcos Luis Ortíz Valmaseda (@marcosluis2186)
> Data Engineer at UCI
> http://marcosluis2186.posterous.com
>
>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci

Im trying to test my MapReduce program on more than one node.
and I only have one computer that has 2 processors.

________________________________
 From: Michael Segel <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc: MapHadoop <[EMAIL PROTECTED]>; [EMAIL PROTECTED]
Sent: Wednesday, May 9, 2012 2:11 PM
Subject: Re: How to run two data nodes on one pc?
 
Too small a machine.

Better question. Why do you want to get 2 nodes on one machine?

On May 9, 2012, at 1:49 PM, Marcos Ortiz wrote:

> Which is your current hardware configuration?
> 1- One way is to use two VMs in Vmware Workstation, Vmware Server, or Virtualbox, with similar configuration for these two DNs, but
> you should have in mind that HBase needs a lot of RAM to work properly, so I don't recommend to use a single machine to run two DN.
>
>
> On 05/09/2012 02:17 PM, MapHadoop wrote:
>> Hi,
>> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
>> What do I have to do to add the other node as a slave?
>
> --
> Marcos Luis Ortíz Valmaseda (@marcosluis2186)
> Data Engineer at UCI
> http://marcosluis2186.posterous.com
>
>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci

You can run two Datanodes on the same machine but it does not prove
much. It is like using two browsers instead of using tabs.

You will have to tween many configuration properties to ensure they do
not overlap on directories or ports.

"By add the other node as a slave" I guess you mean list it twice in
the slaves file. I would just avoid using the slave file all together.

Edward

On Wed, May 9, 2012 at 2:17 PM, MapHadoop <[EMAIL PROTECTED]> wrote:
>
> Hi,
> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
> What do I have to do to add the other node as a slave?
> --
> View this message in context: http://old.nabble.com/How-to-run-two-data-nodes-on-one-pc--tp33763649p33763649.html
> Sent from the HBase User mailing list archive at Nabble.com.
>

You can run in Standalone mode following the quickstart instructions
(google hadoop standalone or hbase standalone).  The other option is
pseudo-distributed mode where it's the same OS uses a different JavaVM.

I agree with Marcos, either you need a commercial virtualization layer
that lets you run double kernels natively (which on a Duo core--which I
doubt has the memory--won't really be that useful) or VMware.

On 5/9/2012 12:11 PM, Michael Segel wrote:
> Too small a machine.
>
> Better question. Why do you want to get 2 nodes on one machine?
>
> On May 9, 2012, at 1:49 PM, Marcos Ortiz wrote:
>
>> Which is your current hardware configuration?
>> 1- One way is to use two VMs in Vmware Workstation, Vmware Server, or Virtualbox, with similar configuration for these two DNs, but
>> you should have in mind that HBase needs a lot of RAM to work properly, so I don't recommend to use a single machine to run two DN.
>>
>>
>> On 05/09/2012 02:17 PM, MapHadoop wrote:
>>> Hi,
>>> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
>>> What do I have to do to add the other node as a slave?
>>
--
[EMAIL PROTECTED], http://bolcer.org, c: +1.714.928.5476

Best Option Insatll two ubuntu in vm and use that :)

On Thu, May 10, 2012 at 12:49 AM, waled tayiib <[EMAIL PROTECTED]> wrote:

> Im trying to test my MapReduce program on more than one node.
> and I only have one computer that has 2 processors.
>
>
>
> ________________________________
>  From: Michael Segel <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc: MapHadoop <[EMAIL PROTECTED]>; [EMAIL PROTECTED]
> Sent: Wednesday, May 9, 2012 2:11 PM
> Subject: Re: How to run two data nodes on one pc?
>
> Too small a machine.
>
> Better question. Why do you want to get 2 nodes on one machine?
>
> On May 9, 2012, at 1:49 PM, Marcos Ortiz wrote:
>
> > Which is your current hardware configuration?
> > 1- One way is to use two VMs in Vmware Workstation, Vmware Server, or
> Virtualbox, with similar configuration for these two DNs, but
> > you should have in mind that HBase needs a lot of RAM to work properly,
> so I don't recommend to use a single machine to run two DN.
> >
> >
> > On 05/09/2012 02:17 PM, MapHadoop wrote:
> >> Hi,
> >> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
> >> What do I have to do to add the other node as a slave?
> >
> > --
> > Marcos Luis Ortíz Valmaseda (@marcosluis2186)
> > Data Engineer at UCI
> > http://marcosluis2186.posterous.com
> >
> >
> >
> > 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS
> INFORMATICAS...
> > CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
> >
> > http://www.uci.cu
> > http://www.facebook.com/universidad.uci
> > http://www.flickr.com/photos/universidad_uci
>

--
∞
Shashwat Shriparv

Waled,

I've covered running multiple daemons on a single node (or so called
"expanded pseudo-distributed mode"? :)) briefly at
http://search-hadoop.com/m/j2iwg1Re6gk with some configs added.

On Thu, May 10, 2012 at 12:49 AM, waled tayiib <[EMAIL PROTECTED]> wrote:
> Im trying to test my MapReduce program on more than one node.
> and I only have one computer that has 2 processors.
>
>
>
> ________________________________
>  From: Michael Segel <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc: MapHadoop <[EMAIL PROTECTED]>; [EMAIL PROTECTED]
> Sent: Wednesday, May 9, 2012 2:11 PM
> Subject: Re: How to run two data nodes on one pc?
>
> Too small a machine.
>
> Better question. Why do you want to get 2 nodes on one machine?
>
> On May 9, 2012, at 1:49 PM, Marcos Ortiz wrote:
>
>> Which is your current hardware configuration?
>> 1- One way is to use two VMs in Vmware Workstation, Vmware Server, or Virtualbox, with similar configuration for these two DNs, but
>> you should have in mind that HBase needs a lot of RAM to work properly, so I don't recommend to use a single machine to run two DN.
>>
>>
>> On 05/09/2012 02:17 PM, MapHadoop wrote:
>>> Hi,
>>> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
>>> What do I have to do to add the other node as a slave?
>>
>> --
>> Marcos Luis Ortíz Valmaseda (@marcosluis2186)
>> Data Engineer at UCI
>> http://marcosluis2186.posterous.com
>>
>>
>>
>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>
>> http://www.uci.cu
>> http://www.facebook.com/universidad.uci
>> http://www.flickr.com/photos/universidad_uci

--
Harsh J

Thank you very much. That is very useful. But for the run-additionalDN.sh file in  this link  http://search-hadoop.com/m/a4klk28NUr12, what did you do to fix it since you got an error.

and i keep getting the error "shift: 24 : Cant shift that many" 

Also for the config files in this link   
https://gist.github.com/2345300
How do i find I find the localhost address for my computer. Or do I use the same numbers they use. For example: localhost:50030
Thanks again for your help

________________________________
 From: Harsh J <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]; waled tayiib <[EMAIL PROTECTED]>
Cc: Michael Segel <[EMAIL PROTECTED]>; "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
Sent: Thursday, May 10, 2012 7:56 AM
Subject: Re: How to run two data nodes on one pc?
 
Waled,

I've covered running multiple daemons on a single node (or so called
"expanded pseudo-distributed mode"? :)) briefly at
http://search-hadoop.com/m/j2iwg1Re6gk with some configs added.

On Thu, May 10, 2012 at 12:49 AM, waled tayiib <[EMAIL PROTECTED]> wrote:
> Im trying to test my MapReduce program on more than one node.
> and I only have one computer that has 2 processors.
>
>
>
> ________________________________
>  From: Michael Segel <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]
> Cc: MapHadoop <[EMAIL PROTECTED]>; [EMAIL PROTECTED]
> Sent: Wednesday, May 9, 2012 2:11 PM
> Subject: Re: How to run two data nodes on one pc?
>
> Too small a machine.
>
> Better question. Why do you want to get 2 nodes on one machine?
>
> On May 9, 2012, at 1:49 PM, Marcos Ortiz wrote:
>
>> Which is your current hardware configuration?
>> 1- One way is to use two VMs in Vmware Workstation, Vmware Server, or Virtualbox, with similar configuration for these two DNs, but
>> you should have in mind that HBase needs a lot of RAM to work properly, so I don't recommend to use a single machine to run two DN.
>>
>>
>> On 05/09/2012 02:17 PM, MapHadoop wrote:
>>> Hi,
>>> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
>>> What do I have to do to add the other node as a slave?
>>
>> --
>> Marcos Luis Ortíz Valmaseda (@marcosluis2186)
>> Data Engineer at UCI
>> http://marcosluis2186.posterous.com
>>
>>
>>
>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>
>> http://www.uci.cu
>> http://www.facebook.com/universidad.uci
>> http://www.flickr.com/photos/universidad_uci

--
Harsh J

Waled,

Since its the same machine, using "localhost" (which refers
automatically to 127.0.0.1) and those explicit port numbers should be
fine to use.

For the DN startup, see my last mail in that link I sent you for an
alternative approach instead of using that script.

On Thu, May 10, 2012 at 11:49 PM, waled tayiib <[EMAIL PROTECTED]> wrote:
>
>
> Thank you very much. That is very useful. But for the run-additionalDN.sh file in  this link  http://search-hadoop.com/m/a4klk28NUr12, what did you do to fix it since you got an error.
>
> and i keep getting the error "shift: 24 : Cant shift that many"
>
> Also for the config files in this link
> https://gist.github.com/2345300
> How do i find I find the localhost address for my computer. Or do I use the same numbers they use. For example: localhost:50030
>
>
> Thanks again for your help
>
> ________________________________
>  From: Harsh J <[EMAIL PROTECTED]>
> To: [EMAIL PROTECTED]; waled tayiib <[EMAIL PROTECTED]>
> Cc: Michael Segel <[EMAIL PROTECTED]>; "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Thursday, May 10, 2012 7:56 AM
> Subject: Re: How to run two data nodes on one pc?
>
> Waled,
>
> I've covered running multiple daemons on a single node (or so called
> "expanded pseudo-distributed mode"? :)) briefly at
> http://search-hadoop.com/m/j2iwg1Re6gk with some configs added.
>
> On Thu, May 10, 2012 at 12:49 AM, waled tayiib <[EMAIL PROTECTED]> wrote:
>> Im trying to test my MapReduce program on more than one node.
>> and I only have one computer that has 2 processors.
>>
>>
>>
>> ________________________________
>>  From: Michael Segel <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Cc: MapHadoop <[EMAIL PROTECTED]>; [EMAIL PROTECTED]
>> Sent: Wednesday, May 9, 2012 2:11 PM
>> Subject: Re: How to run two data nodes on one pc?
>>
>> Too small a machine.
>>
>> Better question. Why do you want to get 2 nodes on one machine?
>>
>> On May 9, 2012, at 1:49 PM, Marcos Ortiz wrote:
>>
>>> Which is your current hardware configuration?
>>> 1- One way is to use two VMs in Vmware Workstation, Vmware Server, or Virtualbox, with similar configuration for these two DNs, but
>>> you should have in mind that HBase needs a lot of RAM to work properly, so I don't recommend to use a single machine to run two DN.
>>>
>>>
>>> On 05/09/2012 02:17 PM, MapHadoop wrote:
>>>> Hi,
>>>> Im trying to get Duo core computer to run 2 data nodes on Hadoop.
>>>> What do I have to do to add the other node as a slave?
>>>
>>> --
>>> Marcos Luis Ortíz Valmaseda (@marcosluis2186)
>>> Data Engineer at UCI
>>> http://marcosluis2186.posterous.com
>>>
>>>
>>>
>>> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
>>> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>>>
>>> http://www.uci.cu
>>> http://www.facebook.com/universidad.uci
>>> http://www.flickr.com/photos/universidad_uci
>
>
>
> --
> Harsh J

--
Harsh J