Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Hbase- Hadoop DFS size not decreased even after deleting a column family


Copy link to this message
-
Re: Hbase- Hadoop DFS size not decreased even after deleting a column family
Hi Stack,
we have got a lot of these in the logs after we deleted the column family.

org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family ol does not exist in region <region>
        at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:4759)
        at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamilies(HRegion.java:2639)
        at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2065)
        at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:1962)
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3428)
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
        at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1386)
2012-08-05 00:11:37,697 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
rollbackMemstore rolled back 0 keyvalues from start:0 to end:1
On Tue, Aug 7, 2012 at 10:27 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looking at TableDeleteFamilyHandler, I see:
>
>   protected void handleTableOperation(List<HRegionInfo> hris) throws
> IOException {
>     // Update table descriptor in HDFS
>     HTableDescriptor htd >       this.masterServices.getMasterFileSystem().deleteColumn(tableName,
> familyName);
>     // Update in-memory descriptor cache
>     this.masterServices.getTableDescriptors().add(htd);
>
> MasterFileSystem.deleteColumn() calls:
>
>     HTableDescriptor htd > this.services.getTableDescriptors().get(tableName);
>     htd.removeFamily(familyName);
>
> I will dig some more.
>
> But looks like there is a bug.
>
> On Tue, Aug 7, 2012 at 9:21 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > On Tue, Aug 7, 2012 at 9:59 AM, prem yadav <[EMAIL PROTECTED]> wrote:
> > > Hi Stack,
> > > yes . I can still see the column family.
> > >
> > > we ran the following command too:
> > >
> > > fs -dus /hbase/<table name>/<region hash>/<cf that was deleted> and it
> > > returned a large value which means that the space has not been
> reclaimed.
> > > The hbase version we are running is 0.94.1.
> > >
> > > is there  a manual way to force reclaim the space?
> > >
> >
> > You could delete the column family from hdfs under each region.
> > Anything in the logs around the remove of the column family?  Maybe we
> > just don't do the delete cleanup in hdfs when we drop a cf (I've not
> > checked)?
> >
> > St.Ack
> >
>