Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Hbase- Hadoop DFS size not decreased even after deleting a column family


Copy link to this message
-
Re: Hbase- Hadoop DFS size not decreased even after deleting a column family
Hi Stack,
we have got a lot of these in the logs after we deleted the column family.

org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family ol does not exist in region <region>
        at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:4759)
        at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamilies(HRegion.java:2639)
        at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2065)
        at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:1962)
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3428)
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
        at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1386)
2012-08-05 00:11:37,697 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
rollbackMemstore rolled back 0 keyvalues from start:0 to end:1
On Tue, Aug 7, 2012 at 10:27 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looking at TableDeleteFamilyHandler, I see:
>
>   protected void handleTableOperation(List<HRegionInfo> hris) throws
> IOException {
>     // Update table descriptor in HDFS
>     HTableDescriptor htd >       this.masterServices.getMasterFileSystem().deleteColumn(tableName,
> familyName);
>     // Update in-memory descriptor cache
>     this.masterServices.getTableDescriptors().add(htd);
>
> MasterFileSystem.deleteColumn() calls:
>
>     HTableDescriptor htd > this.services.getTableDescriptors().get(tableName);
>     htd.removeFamily(familyName);
>
> I will dig some more.
>
> But looks like there is a bug.
>
> On Tue, Aug 7, 2012 at 9:21 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > On Tue, Aug 7, 2012 at 9:59 AM, prem yadav <[EMAIL PROTECTED]> wrote:
> > > Hi Stack,
> > > yes . I can still see the column family.
> > >
> > > we ran the following command too:
> > >
> > > fs -dus /hbase/<table name>/<region hash>/<cf that was deleted> and it
> > > returned a large value which means that the space has not been
> reclaimed.
> > > The hbase version we are running is 0.94.1.
> > >
> > > is there  a manual way to force reclaim the space?
> > >
> >
> > You could delete the column family from hdfs under each region.
> > Anything in the logs around the remove of the column family?  Maybe we
> > just don't do the delete cleanup in hdfs when we drop a cf (I've not
> > checked)?
> >
> > St.Ack
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB