I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work.  I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.

The good news is that all three tasktrackers show up.
The bad news is that only one datanode shows up.

And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.

So, my question is "How do I resolve the collision of the storageIDs?"

Thanks!
-mgl

Hi,

To solve that simply do the following on the problematic nodes:
1) Stop the datanode (probably not running)
2) Remove everything inside the .../cache/hdfs/
3) Start the datanode again.

Note: With cloudera always use "service" way to stop/start hadoop software!
service hadoop-0.20-datanode stop

2011/3/24 Marc Leavitt <[EMAIL PROTECTED]>:
> I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work.  I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.
>
> The good news is that all three tasktrackers show up.
> The bad news is that only one datanode shows up.
>
> And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.
>
> So, my question is "How do I resolve the collision of the storageIDs?"
>
> Thanks!
> -mgl
>
>
>
>
>

--
Met vriendelijke groeten,

Niels Basjes

Worked perfectly.

Thanks Niels!

-mgl

On Mar 24, 2011, at 12:48 PM, Niels Basjes wrote:

> Hi,
>
> To solve that simply do the following on the problematic nodes:
> 1) Stop the datanode (probably not running)
> 2) Remove everything inside the .../cache/hdfs/
> 3) Start the datanode again.
>
> Note: With cloudera always use "service" way to stop/start hadoop software!
> service hadoop-0.20-datanode stop
>
> 2011/3/24 Marc Leavitt <[EMAIL PROTECTED]>:
>> I am setting up a (very) small Hadoop/CDH3 beta 4 cluster in virtual machines to do some initial feasibility work.  I proceeded by progressing through the Cloudera documentation standalone -> pseudo-cluster -> cluster with a single VM and then, when I had it stable(-ish) I copied the VM to a couple of slave images.
>>
>> The good news is that all three tasktrackers show up.
>> The bad news is that only one datanode shows up.
>>
>> And, after after "some" research, I am pretty sure I know why - each of the datanodes is trying to claim the same storageID (as defined in .../cache/hdfs/dfs/data/current/VERSION.
>>
>> So, my question is "How do I resolve the collision of the storageIDs?"
>>
>> Thanks!
>> -mgl
>>
>>
>>
>>
>>
>
>
>
> --
> Met vriendelijke groeten,
>
> Niels Basjes