Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Programmatically Gettting a TServer address for ThriftUtil.getClient?


Copy link to this message
-
Re: Programmatically Gettting a TServer address for ThriftUtil.getClient?
On Thu, Jan 31, 2013 at 9:48 AM, David Medinets
<[EMAIL PROTECTED]> wrote:
> I have some code that totals the number of entries in each tablet will
> the end goal of find the average number of entries per tablet;
> basically for a hotspot report. My code is working on my single-node
> Accumulo but not on the edge node of my Accumulo cluster. On my edge
> node, the TServer list has zero entries.
>
>   1) Any idea why the multi-node cluster is not reporting the set of TServers?
>   2) Is there a different, better, recommended, way to accomplish my goal?

Another way to accomplish this is by scanning the metadata table.
The value of file entries for a tablet contain an estimated size and
estimated number of entries.   You will not know how many entries the
tablet has in memory.  But you can tell if a tablet has entries in
memory or not based on the presences of log entries for the tablet.

I was thinking of writing a little utility that would gather info like
this about a tablet by scanning the metadata table.  I wanted stats
about # of files, file sizes, and # of entries per tablet.  The stats
I was interested in were avg, min, max, and stddev.   Also see
ACCUMULO-397, it mentions ContinuousStatsCollector.getTabletStats().
You could take a look at that.

Keith

>
>
>   LiveTServerSet tserverSet = new LiveTServerSet(instance, new
> DoNothingLiveTServerSetCallback());
>   tserverSet.scanServers();
>   Set<TServerInstance> tserverList = tserverSet.getCurrentServers();
>   if (tserverList.size() == 0) {
>     throw new RuntimeException("NO TSERVERS!");
>   }
>   TServerInstance tServerInstance (TServerInstance) tserverList.toArray()[0];
>   InetSocketAddress tserverAddress > AddressUtil.parseAddress(tServerInstance.host() + ":" +
> tServerInstance.port(), -1);
>
>   List<TabletStats> tsStats = new ArrayList<TabletStats>();
>
>   TableClientService.Iface client = ThrifUtil.getClient(new
> TabletClientService.Client.Factory(), tserverAddress,
> ServerConfiguration.getSystemConfiguration());
>   for (String tableId : mmi.tableMap.keySet()) {
>     tsStats.addAll(client.getTabletStats(null,
> SecurityConstants.getSystemCredentials(), tableId));
>   }
>
>   long totalEntries = 0;
>   long tabletCount = 0;
>   for (TableStats info : tsStats) {
>     totalEntries += info.numEntries;
>     tabletCount++;
>   }