Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Block location in HDFS


Copy link to this message
-
Re: Block location in HDFS
Harsh J 2012-07-04, 16:09
Right, but do you have a specific 4 nodes in mind when you do the
write? If not, you may as well write to HDFS from a non-DN node and
the block locations will be chosen randomly (with the default policy).
This doesn't _guarantee_ that you get different locations for each
block allocation, but does make the placement random.

Otherwise, I suppose HDFS-385 is what you need, and your placement
class can probably just do a round robin assignment of blocks every
time.

Just curious - why do you seek this?

On Wed, Jul 4, 2012 at 8:31 PM, Giulio D'Ippolito
<[EMAIL PROTECTED]> wrote:
> Thanks for the answer,
>
> My problem is slightly different, let say that I have 6 storage point and i
> have 4 files/blocks to insert in the filesystem, i would like that these
> blocks will be placed in 4 different storage points, and of course i would
> like to do the same for all the next 4 blocks.  I'm not interested to the
> replicas for the moment.
>  Thanks
>
>
> Giulio
>
>
>
> 2012/7/4 Harsh J <[EMAIL PROTECTED]>
>>
>> With the default replication/block placement policy, all replicas are
>> anyway kept in separate nodes. Is that what you want?
>>
>> Or are you looking to specify a list of locations you want your blocks
>> to go to? For the latter, there's no easy way but you may be
>> interested in https://issues.apache.org/jira/browse/HDFS-385. But you
>> can't send meta-info with each write with hints even with HDFS-385,
>> and may need an external system.
>>
>> On Wed, Jul 4, 2012 at 4:07 PM, Giulio D'Ippolito
>> <[EMAIL PROTECTED]> wrote:
>> > Hello,
>> >
>> > I would like to insert in the filesystem some blocks, but each of these
>> > blocks must be in a different storage point is there a way to specify
>> > this?
>> >
>> > Thanks
>>
>>
>>
>> --
>> Harsh J
>
>

--
Harsh J