Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Hbase- Hadoop DFS size not decreased even after deleting a column family


+
prem yadav 2012-08-07, 06:19
+
Stack 2012-08-07, 07:22
+
prem yadav 2012-08-07, 08:59
+
Stack 2012-08-07, 16:21
+
Ted Yu 2012-08-07, 16:57
Copy link to this message
-
Re: Hbase- Hadoop DFS size not decreased even after deleting a column family
Hi Stack,
we have got a lot of these in the logs after we deleted the column family.

org.apache.hadoop.hbase.regionserver.NoSuchColumnFamilyException: Column
family ol does not exist in region <region>
        at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamily(HRegion.java:4759)
        at
org.apache.hadoop.hbase.regionserver.HRegion.checkFamilies(HRegion.java:2639)
        at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2065)
        at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:1962)
        at
org.apache.hadoop.hbase.regionserver.HRegionServer.multi(HRegionServer.java:3428)
        at sun.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
        at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:601)
        at
org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:364)
        at
org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1386)
2012-08-05 00:11:37,697 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
rollbackMemstore rolled back 0 keyvalues from start:0 to end:1
On Tue, Aug 7, 2012 at 10:27 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Looking at TableDeleteFamilyHandler, I see:
>
>   protected void handleTableOperation(List<HRegionInfo> hris) throws
> IOException {
>     // Update table descriptor in HDFS
>     HTableDescriptor htd >       this.masterServices.getMasterFileSystem().deleteColumn(tableName,
> familyName);
>     // Update in-memory descriptor cache
>     this.masterServices.getTableDescriptors().add(htd);
>
> MasterFileSystem.deleteColumn() calls:
>
>     HTableDescriptor htd > this.services.getTableDescriptors().get(tableName);
>     htd.removeFamily(familyName);
>
> I will dig some more.
>
> But looks like there is a bug.
>
> On Tue, Aug 7, 2012 at 9:21 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > On Tue, Aug 7, 2012 at 9:59 AM, prem yadav <[EMAIL PROTECTED]> wrote:
> > > Hi Stack,
> > > yes . I can still see the column family.
> > >
> > > we ran the following command too:
> > >
> > > fs -dus /hbase/<table name>/<region hash>/<cf that was deleted> and it
> > > returned a large value which means that the space has not been
> reclaimed.
> > > The hbase version we are running is 0.94.1.
> > >
> > > is there  a manual way to force reclaim the space?
> > >
> >
> > You could delete the column family from hdfs under each region.
> > Anything in the logs around the remove of the column family?  Maybe we
> > just don't do the delete cleanup in hdfs when we drop a cf (I've not
> > checked)?
> >
> > St.Ack
> >
>
+
J Mohamed Zahoor 2012-08-09, 09:03
+
lars hofhansl 2012-08-09, 18:30
+
J Mohamed Zahoor 2012-08-10, 07:17
+
J Mohamed Zahoor 2012-08-10, 07:24
+
J Mohamed Zahoor 2012-08-10, 07:36
+
J Mohamed Zahoor 2012-08-13, 05:43
+
jmozah 2012-08-18, 16:19
+
J Mohamed Zahoor 2012-08-09, 09:35
+
J Mohamed Zahoor 2012-08-09, 10:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB