Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> getMasterStats problems


+
Jim Klucar 2012-07-16, 02:52
+
Adam Fuchs 2012-07-16, 03:09
+
Eric Newton 2012-07-16, 12:08
Copy link to this message
-
Re: getMasterStats problems
Looks like I didn't have the hadoop configs on my classpath, but the
accumulo-site.xml was on the classpath. Once I added the hadoop files,
I was able to create a ZooKeeperInstance, get the system auths, and
the MasterStatus.

I did this in Clojure/Leiningen though and it looks like Clojure is
somehow evaluating static classes / methods etc at compile time, so I
have to have accumulo running to get it to work. This is probably more
of a quirk of Clojure than anything else, but some of the classes
referenced statically use the HdfsZooInstance in them, so that has to
be working for me to compile. weird.

I'll give Adam's route a try and report how it goes.

On Mon, Jul 16, 2012 at 8:08 AM, Eric Newton <[EMAIL PROTECTED]> wrote:
> You also need to have the hadoop configuration in your classpath so
> the client can find HDFS, and not the local file system.
>
> -Eric
>
> On Sun, Jul 15, 2012 at 11:09 PM, Adam Fuchs <[EMAIL PROTECTED]> wrote:
>> Jim,
>>
>> The HdfsZooInstance looks for accumulo-site.xml on the classpath to find the
>> directory in HDFS to look for the instance ID. If accumulo-site.xml is not
>> on the classpath then it will default to /accumulo, which is probably
>> different from the directory you are using. accumulo-site.xml also includes
>> a shared secret which is used to construct the system credentials. This is
>> the standard way that servers communicate with each other.
>>
>> Clients, however, typically don't use HdfsZooInstance. You might try
>> replacing that with ZooKeeperInstance, and then construct your own AuthInfo
>> object (i.e. new
>> AuthInfo("user",ByteBuffer.wrap("password".getBytes()),zkInstance.getInstanceID())
>> ) instead of trying to use the system credentials.
>>
>> Cheers,
>> Adam
>>
>>
>> On Sun, Jul 15, 2012 at 10:52 PM, Jim Klucar <[EMAIL PROTECTED]> wrote:
>>>
>>> I'm trying to get the master stats in the same way the monitor server
>>> does, but I keep having issues.
>>>
>>> The code I'm using is from the GetMasterStats.java program:
>>>
>>>     MasterClientService.Iface client = null;
>>>     MasterMonitorInfo stats = null;
>>>     try {
>>>       client >>> MasterClient.getConnectionWithRetry(HdfsZooInstance.getInstance());
>>>       stats = client.getMasterStats(null,
>>> SecurityConstants.getSystemCredentials());
>>>     } finally {
>>>       if (client != null)
>>>         MasterClient.close(client);
>>>     }
>>>
>>> The problem is that the HdfsZooInstance can never find the instance
>>> data, even though it's there. I can create a ZooKeeperInstance just
>>> fine, but passing that into the MasterClient fails the same way.
>>>
>>> Caused by: java.lang.RuntimeException: Accumulo not initialized, there
>>> is no instance id at /accumulo/instance_id
>>>         at
>>> org.apache.accumulo.core.client.ZooKeeperInstance.getInstanceIDFromHdfs(ZooKeeperInstance.java:263)
>>>         at
>>> org.apache.accumulo.server.conf.ZooConfiguration.getInstance(ZooConfiguration.java:65)
>>>         at
>>> org.apache.accumulo.server.conf.ServerConfiguration.getZooConfiguration(ServerConfiguration.java:49)
>>>         at
>>> org.apache.accumulo.server.conf.ServerConfiguration.getSystemConfiguration(ServerConfiguration.java:58)
>>>         at
>>> org.apache.accumulo.server.client.HdfsZooInstance.<init>(HdfsZooInstance.java:62)
>>>         at
>>> org.apache.accumulo.server.client.HdfsZooInstance.getInstance(HdfsZooInstance.java:70)
>>>         at
>>> org.apache.accumulo.server.security.SecurityConstants.makeSystemPassword(SecurityConstants.java:58)
>>>         at
>>> org.apache.accumulo.server.security.SecurityConstants.<clinit>(SecurityConstants.java:43)
>>>         ... 91 more
>>>
>>> Any ideas as to what I'm missing?
>>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB