Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> Programmatically Gettting a TServer address for ThriftUtil.getClient?


+
David Medinets 2013-01-31, 14:48
+
Keith Turner 2013-01-31, 15:05
+
Eric Newton 2013-01-31, 15:11
Copy link to this message
-
Re: Programmatically Gettting a TServer address for ThriftUtil.getClient?
On Thu, Jan 31, 2013 at 9:48 AM, David Medinets
<[EMAIL PROTECTED]> wrote:
> I have some code that totals the number of entries in each tablet will
> the end goal of find the average number of entries per tablet;
> basically for a hotspot report. My code is working on my single-node
> Accumulo but not on the edge node of my Accumulo cluster. On my edge
> node, the TServer list has zero entries.
>
>   1) Any idea why the multi-node cluster is not reporting the set of TServers?
>   2) Is there a different, better, recommended, way to accomplish my goal?

Another way to accomplish this is by scanning the metadata table.
The value of file entries for a tablet contain an estimated size and
estimated number of entries.   You will not know how many entries the
tablet has in memory.  But you can tell if a tablet has entries in
memory or not based on the presences of log entries for the tablet.

I was thinking of writing a little utility that would gather info like
this about a tablet by scanning the metadata table.  I wanted stats
about # of files, file sizes, and # of entries per tablet.  The stats
I was interested in were avg, min, max, and stddev.   Also see
ACCUMULO-397, it mentions ContinuousStatsCollector.getTabletStats().
You could take a look at that.

Keith

>
>
>   LiveTServerSet tserverSet = new LiveTServerSet(instance, new
> DoNothingLiveTServerSetCallback());
>   tserverSet.scanServers();
>   Set<TServerInstance> tserverList = tserverSet.getCurrentServers();
>   if (tserverList.size() == 0) {
>     throw new RuntimeException("NO TSERVERS!");
>   }
>   TServerInstance tServerInstance (TServerInstance) tserverList.toArray()[0];
>   InetSocketAddress tserverAddress > AddressUtil.parseAddress(tServerInstance.host() + ":" +
> tServerInstance.port(), -1);
>
>   List<TabletStats> tsStats = new ArrayList<TabletStats>();
>
>   TableClientService.Iface client = ThrifUtil.getClient(new
> TabletClientService.Client.Factory(), tserverAddress,
> ServerConfiguration.getSystemConfiguration());
>   for (String tableId : mmi.tableMap.keySet()) {
>     tsStats.addAll(client.getTabletStats(null,
> SecurityConstants.getSystemCredentials(), tableId));
>   }
>
>   long totalEntries = 0;
>   long tabletCount = 0;
>   for (TableStats info : tsStats) {
>     totalEntries += info.numEntries;
>     tabletCount++;
>   }
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB