Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Adding additional storage


Copy link to this message
-
Re: Adding additional storage
Keith Wiley 2012-08-27, 21:52
That appears to have worked.  Thanks.

On Aug 27, 2012, at 11:52 , Harsh J wrote:

> Hey Keith,
>
> Pseudo-distributed isn't any different from fully-distributed,
> operationally, except for nodes = 1 - so don't let it limit your
> thoughts :)
>
> Stop the HDFS cluster, mv your existing dfs.name.dir and dfs.data.dir
> dir contents onto the new storage mount. Reconfigure dfs.data.dir and
> dfs.name.dir to point to these new locations and start it back up. All
> should be well.
>
> On Tue, Aug 28, 2012 at 12:15 AM, Keith Wiley <[EMAIL PROTECTED]> wrote:
>> I'm running a pseudo-distributed cluster on a single machine and I would like to use a larger disk (mounted and ready to go of course).  I don't mind transferring to the new disk (as opposed to using both disks for the hdfs which seems much hairier), but I'm not sure how to transfer a hadoop cluster to a new disk...or if its even possible.  Even if I simply copy the entire directory where the hdfs is emulated, I still need to somehow switch the namenode to know about the new disk?
>>
>> Is there any way to do this that beats manually "getting" the data, throwing away the old cluster, making a new cluster from scratch, and reuploading the data to hdfs?...or is that really the only feasible way to migrate a pseudo-distributed cluster to a second larger storage?
>>
>> Thanks.
>>
>> ________________________________________________________________________________
>> Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com
>>
>> "You can scratch an itch, but you can't itch a scratch. Furthermore, an itch can
>> itch but a scratch can't scratch. Finally, a scratch can itch, but an itch can't
>> scratch. All together this implies: He scratched the itch from the scratch that
>> itched but would never itch the scratch from the itch that scratched."
>>                                           --  Keith Wiley
>> ________________________________________________________________________________
>>
>
>
>
> --
> Harsh J
________________________________________________________________________________
Keith Wiley     [EMAIL PROTECTED]     keithwiley.com    music.keithwiley.com

"Luminous beings are we, not this crude matter."
                                           --  Yoda
________________________________________________________________________________