Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # dev - Is there a way to get a Block through block id?


Copy link to this message
-
Re: Is there a way to get a Block through block id?
Colin McCabe 2014-01-30, 03:30
You could look at the BlocksMap.  That is where blocks should reside.  It
depends on what you're trying to do.

cheers,
Colin
On Tue, Jan 21, 2014 at 10:00 PM, Yu Li <[EMAIL PROTECTED]> wrote:

> Hi Colin,
>
> Thanks for the reply. I guess you're referring to the below methods? If so,
> I'm afraid it can only get an empty block but not the real block I really
> want to move.
> =====================================
>   public ExtendedBlock(final String poolId, final long blockId) {
>     this(poolId, blockId, 0, 0);
>   }
>   public ExtendedBlock(final String poolId, final long blkid, final long
> len,
>       final long genstamp) {
>     this.poolId = poolId;
>     block = new Block(blkid, len, genstamp);
>   }
> =====================================
>
> I'm now working around this by adding an option to directly move the block
> during fsck (NmaenodeFsck). But still, if there's any method to directly
> get the Block instance by blockid through any api, it would be great to
> know. :-)
>
> On 21 January 2014 03:46, Colin McCabe <[EMAIL PROTECTED]> wrote:
>
> > In order to uniquely identify a block in hadoop 2.2, you are going to
> need
> > both a block and a block pool ID.  You can construct a Block object with
> > those two items.
> >
> > On Wed, Jan 15, 2014 at 8:46 AM, Yu Li <[EMAIL PROTECTED]> wrote:
> >
> > > Dear all,
> > >
> > > As titled, I actually have two questions here:
> > >
> > > 1. In current releases like hadoop-2.2.0, is block id unique and able
> to
> > > locate a Block in HDFS? I'm asking because I could see HDFS-4645 is
> > trying
> > > to resolve the uniqueness issue. However, from the code comment it
> seems
> > > block id is expected to be unique
> > >
> > >
> > It's expected to be unique within a block pool.  You can get the block
> pool
> > ID you are using from your FSNamesystem object in 2.2.
> >
> > best,
> > Colin
> >
> > 2. It seems there's no method to get a Block object through a block id.
> > > However, there's some scenarios need such method, like if I use the
> > > FavouredNode feature to create logical datanode group and planned to
> put
> > > some data within a group, then I might need to periodically check
> whether
> > > there's block somehow placed outside the group, and move it back. In
> such
> > > scenario, I would need to first get block ids, then move. But to move
> > them,
> > > it seems we need a Block instance to initiate the ExtendedBlock object.
> > >
> > > Any suggestion, or reference to existing JIRA would be highly
> > appreciated,
> > > and thanks in advance!
> > >
> > > --
> > > Best Regards,
> > > Li Yu
> > >
> >
>
>
>
> --
> Best Regards,
> Li Yu
>