Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Truncate tables


Copy link to this message
-
Re: Truncate tables
It is strage thing in hbase. Operations like create or drop are asyncronous,
so immidiately after first rpc 'disable'
hbase client try to check succesfull execution. Often it is not really
complete yet, so hbase client pauses an
amount of time configured in 'hbase.client.pause'. If you change it in you
Configuration before connect to hbase to,
for example 1000 you will super fast drop/disable/create cycle. But bevare,
you should increment amount of tries,
configured with 'hbase.client.retries.number', or you can fail, if real
disable takes longer then 10 seconds (10 tryes by 1 second each).

2011/2/14 Peter Haidinyak <[EMAIL PROTECTED]>

> Each table is taking 30 seconds. Without looking at the HBase code I
> assumed it was doing the same process as I am. Just quicker.
>
> Thanks
>
> -Pete
>
> -----Original Message-----
> From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED]] On Behalf Of
> Jean-Daniel Cryans
> Sent: Monday, February 14, 2011 10:12 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Truncate tables
>
> Erm well could it be that your script is truncating a bunch of tables
> whereas in the shell it only does one? Apart from that I can't think
> of anything obvious, the truncate command really does the same thing
> as you.
>
> J-D
>
> On Mon, Feb 14, 2011 at 9:43 AM, Peter Haidinyak <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >        I've setup a little utility to truncate my tables during
> development but I found that if I truncate a table from the HBase shell it
> takes 2.5 seconds, from my program it take 30 seconds. The code to truncate
> is pretty simple. Any ideas on what I've done wrong?
> >
> > Thanks
> >
> > -Pete
> >
> >  private void truncateTables()
> >    throws IOException
> >  {
> >    m_hbaseConnection = new HbaseConnection();
> >    m_hbaseConnection.setupHBaseConnection();
> >    final HBaseAdmin hBaseAdmin = m_hbaseConnection.getAdmin();
> >
> >    int counter = 1;
> >    for (final String tableName : CLUSTER_TABLES)
> >    {
> >      final HTableDescriptor hTableDescriptor > hBaseAdmin.getTableDescriptor(tableName.getBytes());
> >      m_logger.info("Truncating Table {}.",tableName);
> >      hBaseAdmin.disableTable(tableName);
> >      hBaseAdmin.deleteTable(tableName);
> >      hBaseAdmin.createTable(hTableDescriptor);
> >      m_logger.info("Truncated Table {}.", tableName);
> >      m_logger.info("Finished {} of {} Tables.", counter++,
> CLUSTER_TABLES.length);
> >
> >    } // - End for each table.
> >
> >  } // - End truncateTables Method.
> >
> >
> >  public void setupHBaseConnection()
> >    throws IOException
> >  {
> >    m_hbaseConfiguration = HBaseConfiguration.create();
> >    m_hbaseConfiguration.clear();
> >
> > // - Get the m_hbaseConfiguration properties from the Command line -D
> parameters.
> >    m_hbaseConfiguration.set("hbase.zookeeper.quorum",
> System.getProperty("hbase.zookeeper.quorum"));
> >    m_hbaseConfiguration.set("hbase.zookeeper.property.clientPort",
> System.getProperty("hbase.zookeeper.property.clientPort"));
> >
> >    m_logger.info("Using hbase.zookeeper.quorum : {}",
> m_hbaseConfiguration.get("hbase.zookeeper.quorum"));
> >    m_logger.info("Using hbase.zookeeper.property.clientPort : {}",
> m_hbaseConfiguration.get("hbase.zookeeper.property.clientPort"));
> >
> >  } // - End setupHBaseConnection Method.
> >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB