Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Block location in HDFS


Copy link to this message
-
Re: Block location in HDFS
Right, but do you have a specific 4 nodes in mind when you do the
write? If not, you may as well write to HDFS from a non-DN node and
the block locations will be chosen randomly (with the default policy).
This doesn't _guarantee_ that you get different locations for each
block allocation, but does make the placement random.

Otherwise, I suppose HDFS-385 is what you need, and your placement
class can probably just do a round robin assignment of blocks every
time.

Just curious - why do you seek this?

On Wed, Jul 4, 2012 at 8:31 PM, Giulio D'Ippolito
<[EMAIL PROTECTED]> wrote:
> Thanks for the answer,
>
> My problem is slightly different, let say that I have 6 storage point and i
> have 4 files/blocks to insert in the filesystem, i would like that these
> blocks will be placed in 4 different storage points, and of course i would
> like to do the same for all the next 4 blocks.  I'm not interested to the
> replicas for the moment.
>  Thanks
>
>
> Giulio
>
>
>
> 2012/7/4 Harsh J <[EMAIL PROTECTED]>
>>
>> With the default replication/block placement policy, all replicas are
>> anyway kept in separate nodes. Is that what you want?
>>
>> Or are you looking to specify a list of locations you want your blocks
>> to go to? For the latter, there's no easy way but you may be
>> interested in https://issues.apache.org/jira/browse/HDFS-385. But you
>> can't send meta-info with each write with hints even with HDFS-385,
>> and may need an external system.
>>
>> On Wed, Jul 4, 2012 at 4:07 PM, Giulio D'Ippolito
>> <[EMAIL PROTECTED]> wrote:
>> > Hello,
>> >
>> > I would like to insert in the filesystem some blocks, but each of these
>> > blocks must be in a different storage point is there a way to specify
>> > this?
>> >
>> > Thanks
>>
>>
>>
>> --
>> Harsh J
>
>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB