Hello,

I followed all of the setup / initialization steps. However, when I
ran accumulo init, it hangs on naming the instance.

mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo init
14 14:15:24,536 [util.Initialize] INFO : Hadoop Filesystem is maprfs:///
14 14:15:24,584 [util.Initialize] INFO : Accumulo data dir is /accumulo
14 14:15:24,585 [util.Initialize] INFO : Zookeeper server is localhost:2181
Instance name : test

Init doesn't seem to produce any logs (at least not in the
accumolo/logs dir), is there a verbose mode or something that can help
me figure out where it is failing?

Thanks,
-Todd

Is zookeeper up and running?

If so can you jstack the init java process and send the stack trace of
the main thread?

Keith

On Wed, Mar 14, 2012 at 5:31 PM, Todd Stavish <[EMAIL PROTECTED]> wrote:
> Hello,
>
> I followed all of the setup / initialization steps. However, when I
> ran accumulo init, it hangs on naming the instance.
>
> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo init
> 14 14:15:24,536 [util.Initialize] INFO : Hadoop Filesystem is maprfs:///
> 14 14:15:24,584 [util.Initialize] INFO : Accumulo data dir is /accumulo
> 14 14:15:24,585 [util.Initialize] INFO : Zookeeper server is localhost:2181
> Instance name : test
>
> Init doesn't seem to produce any logs (at least not in the
> accumolo/logs dir), is there a verbose mode or something that can help
> me figure out where it is failing?
>
> Thanks,
> -Todd

Hi,

You need to specify ZooKeeper client port to 5181 when you use MapR.
MapR's ZooKeeper client port is 5181 by default.

Thanks.
2012/3/15 Keith Turner <[EMAIL PROTECTED]>

> Is zookeeper up and running?
>
> If so can you jstack the init java process and send the stack trace of
> the main thread?
>
> Keith
>
> On Wed, Mar 14, 2012 at 5:31 PM, Todd Stavish <[EMAIL PROTECTED]>
> wrote:
> > Hello,
> >
> > I followed all of the setup / initialization steps. However, when I
> > ran accumulo init, it hangs on naming the instance.
> >
> > mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo init
> > 14 14:15:24,536 [util.Initialize] INFO : Hadoop Filesystem is maprfs:///
> > 14 14:15:24,584 [util.Initialize] INFO : Accumulo data dir is /accumulo
> > 14 14:15:24,585 [util.Initialize] INFO : Zookeeper server is
> localhost:2181
> > Instance name : test
> >
> > Init doesn't seem to produce any logs (at least not in the
> > accumolo/logs dir), is there a verbose mode or something that can help
> > me figure out where it is failing?
> >
> > Thanks,
> > -Todd
>

--
Takuya UESHIN
Tokyo, Japan

http://twitter.com/ueshin

Domo, Takuya-san.  For posterity's sake ->

In accumulo-site.xml:

<property>
  <name>instance.zookeeper.host</name>
  <value>localhost:5181</value>
  <description>comma separated list of zookeeper servers</description>
</property>

On Wed, Mar 14, 2012 at 9:33 PM, Takuya UESHIN <[EMAIL PROTECTED]> wrote:
> Hi,
>
> You need to specify ZooKeeper client port to 5181 when you use MapR.
> MapR's ZooKeeper client port is 5181 by default.
>
> Thanks.
>
>
> 2012/3/15 Keith Turner <[EMAIL PROTECTED]>
>>
>> Is zookeeper up and running?
>>
>> If so can you jstack the init java process and send the stack trace of
>> the main thread?
>>
>> Keith
>>
>> On Wed, Mar 14, 2012 at 5:31 PM, Todd Stavish <[EMAIL PROTECTED]>
>> wrote:
>> > Hello,
>> >
>> > I followed all of the setup / initialization steps. However, when I
>> > ran accumulo init, it hangs on naming the instance.
>> >
>> > mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo init
>> > 14 14:15:24,536 [util.Initialize] INFO : Hadoop Filesystem is maprfs:///
>> > 14 14:15:24,584 [util.Initialize] INFO : Accumulo data dir is /accumulo
>> > 14 14:15:24,585 [util.Initialize] INFO : Zookeeper server is
>> > localhost:2181
>> > Instance name : test
>> >
>> > Init doesn't seem to produce any logs (at least not in the
>> > accumolo/logs dir), is there a verbose mode or something that can help
>> > me figure out where it is failing?
>> >
>> > Thanks,
>> > -Todd
>
>
>
>
> --
> Takuya UESHIN
> Tokyo, Japan
>
> http://twitter.com/ueshin

accumulo init seemed to work. Full disclosure, I am running accumlo in
the MapR M5 vm. I cranked down the memory allocations in
accumulo-env.sh to compensate (maybe too much?). I'm not seeing all of
the process launch. Does the monitor and GC run in their own
processes? Neither the monitoring web page nor accumulo shell are
working. Accumulo shell output below, in addition, ... running
processes, logs, and confs attached.

Running the whole MapR/Accumulo stack all in a single VM is flirting
with oxymoron. My goal was just to go through the install sequence and
run a few samples. If there is little hope in getting this to work in
a vm, please let me now.
mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo shell
Enter current password for 'mapr'@'myinstance': *******
16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an available
server in the list of servers: [mapr-desktop:9999:9999 (120000)]
16 08:41:15,974 [shell.Shell] ERROR:
org.apache.accumulo.core.client.AccumuloException:
org.apache.thrift.transport.TTransportException: Failed to connect to
a server
Here's what's running:

mapr     31166  0.2  3.8 708684 78332 ?        Sl   08:08   0:04
/usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=master -classpath
/opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -Xmx256m
-Xms256m -XX:OnOutOfMemoryError=kill -9 %p
-Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
org.apache.accumulo.start.Main master --address mapr-desktop

mapr     31674  0.2  3.8 703408 78112 ?        Sl   08:08   0:03
/usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=logger -classpath
/opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -Xmx256m
-Xms256m -XX:OnOutOfMemoryError=kill -9 %p
-Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
org.apache.accumulo.start.Main logger --address mapr-desktop

mapr     31853  0.3  4.4 752476 91420 ?        Sl   08:08   0:04
/usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=tserver -classpath
/opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
-XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -Xmx256m
-Xms256m -Xss128k -XX:OnOutOfMemoryError=kill -9 %p
-Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
org.apache.accumulo.start.Main tserver --address mapr-desktop

-Todd
On Thu, Mar 15, 2012 at 4:38 PM, Todd Stavish <[EMAIL PROTECTED]> wrote:
> Domo, Takuya-san.  For posterity's sake ->
>
> In accumulo-site.xml:
>
> <property>
>  <name>instance.zookeeper.host</name>
>  <value>localhost:5181</value>
>  <description>comma separated list of zookeeper servers</description>

The monitor and gc do run in their own processes, which is kinda hinted at
the JVM memory sizes in conf/accumulo-env.sh

The monitor may not be working because your jvm/os security profile is
different from those we've seen to date.
Replace conf/monitor.security.policy with:

 grant{
  permission java.security.AllPermission;
};

Or just chase down all the security violations you are getting in your
jvm/OS.  A quick look at the monitor logs and they seem ok.

Your tserver isn't starting because some other process is listening on the
port.  The monitor can't talk to the master because it's waiting for some
tablet servers to show up.  The monitor may not display until it talks to
the master.  Good news: the logger started up fine.

Do this:

echo localhost > conf/gc

That will get the gc to start up.

Then this:

# kill everything accumulo related
pkill -9 -f accumulo.start

# start everything again
./bin/start-all.sh

Most of these issues, as well as some example configs for specific memory
sizes are fixed in the accumule-1.4.0 release candidate.

-Eric
On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <[EMAIL PROTECTED]>wrote:

> accumulo init seemed to work. Full disclosure, I am running accumlo in
> the MapR M5 vm. I cranked down the memory allocations in
> accumulo-env.sh to compensate (maybe too much?). I'm not seeing all of
> the process launch. Does the monitor and GC run in their own
> processes? Neither the monitoring web page nor accumulo shell are
> working. Accumulo shell output below, in addition, ... running
> processes, logs, and confs attached.
>
> Running the whole MapR/Accumulo stack all in a single VM is flirting
> with oxymoron. My goal was just to go through the install sequence and
> run a few samples. If there is little hope in getting this to work in
> a vm, please let me now.
>
>
> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo shell
> Enter current password for 'mapr'@'myinstance': *******
> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an available
> server in the list of servers: [mapr-desktop:9999:9999 (120000)]
> 16 08:41:15,974 [shell.Shell] ERROR:
> org.apache.accumulo.core.client.AccumuloException:
> org.apache.thrift.transport.TTransportException: Failed to connect to
> a server
>
>
> Here's what's running:
>
> mapr     31166  0.2  3.8 708684 78332 ?        Sl   08:08   0:04
> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=master -classpath
>
> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -Xmx256m
> -Xms256m -XX:OnOutOfMemoryError=kill -9 %p
>
> -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
> org.apache.accumulo.start.Main master --address mapr-desktop
>
> mapr     31674  0.2  3.8 703408 78112 ?        Sl   08:08   0:03
> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=logger -classpath
>
> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -Xmx256m
> -Xms256m -XX:OnOutOfMemoryError=kill -9 %p
>
> -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
> org.apache.accumulo.start.Main logger --address mapr-desktop

Thanks Eric. netstat didn't indicate anyting else was listening at
9999. I switched to 1.4RC2 using confs from 512MB/sandalone. tserver
and master are starting. I don't see the port conflict anymore, but I
am seeing the following exception in all of the logs:

20 09:00:07,644 [tabletserver.TabletServer] ERROR: Uncaught exception
in TabletServer.main, exiting
java.lang.ClassCastException: com.mapr.fs.MapRFileSystem cannot be
cast to org.apache.hadoop.hdfs.DistributedFileSystem
at org.apache.accumulo.server.Accumulo.waitForZookeeperAndHdfs(Accumulo.java:179)
at org.apache.accumulo.server.Accumulo.init(Accumulo.java:130)
at org.apache.accumulo.server.tabletserver.TabletServer.config(TabletServer.java:2939)
at org.apache.accumulo.server.tabletserver.TabletServer.main(TabletServer.java:3109)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.accumulo.start.Main$1.run(Main.java:89)
at java.lang.Thread.run(Thread.java:662)

I'll be at the accumulo meetup tomorrow and can show this live if need be.

-Todd
On Fri, Mar 16, 2012 at 9:19 AM, Eric Newton <[EMAIL PROTECTED]> wrote:
> The monitor and gc do run in their own processes, which is kinda hinted at
> the JVM memory sizes in conf/accumulo-env.sh
>
> The monitor may not be working because your jvm/os security profile is
> different from those we've seen to date.
> Replace conf/monitor.security.policy with:
>
>  grant{
>   permission java.security.AllPermission;
> };
>
> Or just chase down all the security violations you are getting in your
> jvm/OS.  A quick look at the monitor logs and they seem ok.
>
> Your tserver isn't starting because some other process is listening on the
> port.  The monitor can't talk to the master because it's waiting for some
> tablet servers to show up.  The monitor may not display until it talks to
> the master.  Good news: the logger started up fine.
>
> Do this:
>
> echo localhost > conf/gc
>
> That will get the gc to start up.
>
> Then this:
>
> # kill everything accumulo related
> pkill -9 -f accumulo.start
>
> # start everything again
> ./bin/start-all.sh
>
> Most of these issues, as well as some example configs for specific memory
> sizes are fixed in the accumule-1.4.0 release candidate.
>
> -Eric
>
>
> On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <[EMAIL PROTECTED]>
> wrote:
>>
>> accumulo init seemed to work. Full disclosure, I am running accumlo in
>> the MapR M5 vm. I cranked down the memory allocations in
>> accumulo-env.sh to compensate (maybe too much?). I'm not seeing all of
>> the process launch. Does the monitor and GC run in their own
>> processes? Neither the monitoring web page nor accumulo shell are
>> working. Accumulo shell output below, in addition, ... running
>> processes, logs, and confs attached.
>>
>> Running the whole MapR/Accumulo stack all in a single VM is flirting
>> with oxymoron. My goal was just to go through the install sequence and
>> run a few samples. If there is little hope in getting this to work in
>> a vm, please let me now.
>>
>>
>> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo shell
>> Enter current password for 'mapr'@'myinstance': *******
>> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an available
>> server in the list of servers: [mapr-desktop:9999:9999 (120000)]
>> 16 08:41:15,974 [shell.Shell] ERROR:
>> org.apache.accumulo.core.client.AccumuloException:
>> org.apache.thrift.transport.TTransportException: Failed to connect to
>> a server
>>
>>
>> Here's what's running:
>>
>> mapr     31166  0.2  3.8 708684 78332 ?        Sl   08:08   0:04
>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=master -classpath
>>
>> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar

On Tue, Mar 20, 2012 at 12:34 PM, Todd Stavish <[EMAIL PROTECTED]> wrote:
> Thanks Eric. netstat didn't indicate anyting else was listening at
> 9999. I switched to 1.4RC2 using confs from 512MB/sandalone. tserver
> and master are starting. I don't see the port conflict anymore, but I
> am seeing the following exception in all of the logs:
>
> 20 09:00:07,644 [tabletserver.TabletServer] ERROR: Uncaught exception
> in TabletServer.main, exiting
> java.lang.ClassCastException: com.mapr.fs.MapRFileSystem cannot be
> cast to org.apache.hadoop.hdfs.DistributedFileSystem
>        at org.apache.accumulo.server.Accumulo.waitForZookeeperAndHdfs(Accumulo.java:179)
>        at org.apache.accumulo.server.Accumulo.init(Accumulo.java:130)
>        at org.apache.accumulo.server.tabletserver.TabletServer.config(TabletServer.java:2939)
>        at org.apache.accumulo.server.tabletserver.TabletServer.main(TabletServer.java:3109)
>        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>        at java.lang.reflect.Method.invoke(Method.java:597)
>        at org.apache.accumulo.start.Main$1.run(Main.java:89)
>        at java.lang.Thread.run(Thread.java:662)
>
> I'll be at the accumulo meetup tomorrow and can show this live if need be.
>
> -Todd

I looked at the code and its just simply cast to a
DistributedFileSystem inorder to check if HDFS is in safe mode.  It
should do an instanceof operation and then do the case.  Would you
mind opening a ticket about this?

>
>
> On Fri, Mar 16, 2012 at 9:19 AM, Eric Newton <[EMAIL PROTECTED]> wrote:
>> The monitor and gc do run in their own processes, which is kinda hinted at
>> the JVM memory sizes in conf/accumulo-env.sh
>>
>> The monitor may not be working because your jvm/os security profile is
>> different from those we've seen to date.
>> Replace conf/monitor.security.policy with:
>>
>>  grant{
>>   permission java.security.AllPermission;
>> };
>>
>> Or just chase down all the security violations you are getting in your
>> jvm/OS.  A quick look at the monitor logs and they seem ok.
>>
>> Your tserver isn't starting because some other process is listening on the
>> port.  The monitor can't talk to the master because it's waiting for some
>> tablet servers to show up.  The monitor may not display until it talks to
>> the master.  Good news: the logger started up fine.
>>
>> Do this:
>>
>> echo localhost > conf/gc
>>
>> That will get the gc to start up.
>>
>> Then this:
>>
>> # kill everything accumulo related
>> pkill -9 -f accumulo.start
>>
>> # start everything again
>> ./bin/start-all.sh
>>
>> Most of these issues, as well as some example configs for specific memory
>> sizes are fixed in the accumule-1.4.0 release candidate.
>>
>> -Eric
>>
>>
>> On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <[EMAIL PROTECTED]>
>> wrote:
>>>
>>> accumulo init seemed to work. Full disclosure, I am running accumlo in
>>> the MapR M5 vm. I cranked down the memory allocations in
>>> accumulo-env.sh to compensate (maybe too much?). I'm not seeing all of
>>> the process launch. Does the monitor and GC run in their own
>>> processes? Neither the monitoring web page nor accumulo shell are
>>> working. Accumulo shell output below, in addition, ... running
>>> processes, logs, and confs attached.
>>>
>>> Running the whole MapR/Accumulo stack all in a single VM is flirting
>>> with oxymoron. My goal was just to go through the install sequence and
>>> run a few samples. If there is little hope in getting this to work in
>>> a vm, please let me now.
>>>
>>>
>>> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo shell
>>> Enter current password for 'mapr'@'myinstance': *******
>>> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an available
>>> server in the list of servers: [mapr-desktop:9999:9999 (120000)]

> I looked at the code and its just simply cast to a
> DistributedFileSystem inorder to check if HDFS is in safe mode.  It
> should do an instanceof operation and then do the case.  Would you
> mind opening a ticket about this?

Thanks Keith. Done.

https://issues.apache.org/jira/browse/ACCUMULO-476

>
>>
>>
>> On Fri, Mar 16, 2012 at 9:19 AM, Eric Newton <[EMAIL PROTECTED]> wrote:
>>> The monitor and gc do run in their own processes, which is kinda hinted at
>>> the JVM memory sizes in conf/accumulo-env.sh
>>>
>>> The monitor may not be working because your jvm/os security profile is
>>> different from those we've seen to date.
>>> Replace conf/monitor.security.policy with:
>>>
>>>  grant{
>>>   permission java.security.AllPermission;
>>> };
>>>
>>> Or just chase down all the security violations you are getting in your
>>> jvm/OS.  A quick look at the monitor logs and they seem ok.
>>>
>>> Your tserver isn't starting because some other process is listening on the
>>> port.  The monitor can't talk to the master because it's waiting for some
>>> tablet servers to show up.  The monitor may not display until it talks to
>>> the master.  Good news: the logger started up fine.
>>>
>>> Do this:
>>>
>>> echo localhost > conf/gc
>>>
>>> That will get the gc to start up.
>>>
>>> Then this:
>>>
>>> # kill everything accumulo related
>>> pkill -9 -f accumulo.start
>>>
>>> # start everything again
>>> ./bin/start-all.sh
>>>
>>> Most of these issues, as well as some example configs for specific memory
>>> sizes are fixed in the accumule-1.4.0 release candidate.
>>>
>>> -Eric
>>>
>>>
>>> On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <[EMAIL PROTECTED]>
>>> wrote:
>>>>
>>>> accumulo init seemed to work. Full disclosure, I am running accumlo in
>>>> the MapR M5 vm. I cranked down the memory allocations in
>>>> accumulo-env.sh to compensate (maybe too much?). I'm not seeing all of
>>>> the process launch. Does the monitor and GC run in their own
>>>> processes? Neither the monitoring web page nor accumulo shell are
>>>> working. Accumulo shell output below, in addition, ... running
>>>> processes, logs, and confs attached.
>>>>
>>>> Running the whole MapR/Accumulo stack all in a single VM is flirting
>>>> with oxymoron. My goal was just to go through the install sequence and
>>>> run a few samples. If there is little hope in getting this to work in
>>>> a vm, please let me now.
>>>>
>>>>
>>>> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo shell
>>>> Enter current password for 'mapr'@'myinstance': *******
>>>> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an available
>>>> server in the list of servers: [mapr-desktop:9999:9999 (120000)]
>>>> 16 08:41:15,974 [shell.Shell] ERROR:
>>>> org.apache.accumulo.core.client.AccumuloException:
>>>> org.apache.thrift.transport.TTransportException: Failed to connect to
>>>> a server
>>>>
>>>>
>>>> Here's what's running:
>>>>
>>>> mapr     31166  0.2  3.8 708684 78332 ?        Sl   08:08   0:04
>>>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=master -classpath
>>>>
>>>> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
>>>> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -Xmx256m
>>>> -Xms256m -XX:OnOutOfMemoryError=kill -9 %p
>>>>
>>>> -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64
>>>> org.apache.accumulo.start.Main master --address mapr-desktop
>>>>
>>>> mapr     31674  0.2  3.8 703408 78112 ?        Sl   08:08   0:03
>>>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=logger -classpath
>>>>
>>>> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar

Here are the complete instructions for running Accumulo on MapR M5 VM ->

http://t.co/RJJ8Ht4B

Thanks for all of the help,
-Todd
On Tue, Mar 20, 2012 at 1:10 PM, Todd Stavish <[EMAIL PROTECTED]> wrote:
>> I looked at the code and its just simply cast to a
>> DistributedFileSystem inorder to check if HDFS is in safe mode.  It
>> should do an instanceof operation and then do the case.  Would you
>> mind opening a ticket about this?
>
> Thanks Keith. Done.
>
> https://issues.apache.org/jira/browse/ACCUMULO-476
>
>
>
>>
>>>
>>>
>>> On Fri, Mar 16, 2012 at 9:19 AM, Eric Newton <[EMAIL PROTECTED]> wrote:
>>>> The monitor and gc do run in their own processes, which is kinda hinted at
>>>> the JVM memory sizes in conf/accumulo-env.sh
>>>>
>>>> The monitor may not be working because your jvm/os security profile is
>>>> different from those we've seen to date.
>>>> Replace conf/monitor.security.policy with:
>>>>
>>>>  grant{
>>>>   permission java.security.AllPermission;
>>>> };
>>>>
>>>> Or just chase down all the security violations you are getting in your
>>>> jvm/OS.  A quick look at the monitor logs and they seem ok.
>>>>
>>>> Your tserver isn't starting because some other process is listening on the
>>>> port.  The monitor can't talk to the master because it's waiting for some
>>>> tablet servers to show up.  The monitor may not display until it talks to
>>>> the master.  Good news: the logger started up fine.
>>>>
>>>> Do this:
>>>>
>>>> echo localhost > conf/gc
>>>>
>>>> That will get the gc to start up.
>>>>
>>>> Then this:
>>>>
>>>> # kill everything accumulo related
>>>> pkill -9 -f accumulo.start
>>>>
>>>> # start everything again
>>>> ./bin/start-all.sh
>>>>
>>>> Most of these issues, as well as some example configs for specific memory
>>>> sizes are fixed in the accumule-1.4.0 release candidate.
>>>>
>>>> -Eric
>>>>
>>>>
>>>> On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <[EMAIL PROTECTED]>
>>>> wrote:
>>>>>
>>>>> accumulo init seemed to work. Full disclosure, I am running accumlo in
>>>>> the MapR M5 vm. I cranked down the memory allocations in
>>>>> accumulo-env.sh to compensate (maybe too much?). I'm not seeing all of
>>>>> the process launch. Does the monitor and GC run in their own
>>>>> processes? Neither the monitoring web page nor accumulo shell are
>>>>> working. Accumulo shell output below, in addition, ... running
>>>>> processes, logs, and confs attached.
>>>>>
>>>>> Running the whole MapR/Accumulo stack all in a single VM is flirting
>>>>> with oxymoron. My goal was just to go through the install sequence and
>>>>> run a few samples. If there is little hope in getting this to work in
>>>>> a vm, please let me now.
>>>>>
>>>>>
>>>>> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo shell
>>>>> Enter current password for 'mapr'@'myinstance': *******
>>>>> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an available
>>>>> server in the list of servers: [mapr-desktop:9999:9999 (120000)]
>>>>> 16 08:41:15,974 [shell.Shell] ERROR:
>>>>> org.apache.accumulo.core.client.AccumuloException:
>>>>> org.apache.thrift.transport.TTransportException: Failed to connect to
>>>>> a server
>>>>>
>>>>>
>>>>> Here's what's running:
>>>>>
>>>>> mapr     31166  0.2  3.8 708684 78332 ?        Sl   08:08   0:04
>>>>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=master -classpath
>>>>>
>>>>> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar
>>>>> -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -Xmx256m
>>>>> -Xms256m -XX:OnOutOfMemoryError=kill -9 %p
>>>>>
>>>>> -Djava.library.path=/opt/mapr/hadoop/hadoop-0.20.2//lib/native/Linux-amd64-64

Final update, I tested 1.4.0rc5 against MapR v1.2.3. Everything works
well. The only necessary install steps are the port mapping ones. Very
clean.

-Todd
On Tue, Mar 20, 2012 at 9:39 PM, Todd Stavish <[EMAIL PROTECTED]> wrote:
> Here are the complete instructions for running Accumulo on MapR M5 VM ->
>
> http://t.co/RJJ8Ht4B
>
> Thanks for all of the help,
> -Todd
>
>
>
>
> On Tue, Mar 20, 2012 at 1:10 PM, Todd Stavish <[EMAIL PROTECTED]> wrote:
>>> I looked at the code and its just simply cast to a
>>> DistributedFileSystem inorder to check if HDFS is in safe mode.  It
>>> should do an instanceof operation and then do the case.  Would you
>>> mind opening a ticket about this?
>>
>> Thanks Keith. Done.
>>
>> https://issues.apache.org/jira/browse/ACCUMULO-476
>>
>>
>>
>>>
>>>>
>>>>
>>>> On Fri, Mar 16, 2012 at 9:19 AM, Eric Newton <[EMAIL PROTECTED]> wrote:
>>>>> The monitor and gc do run in their own processes, which is kinda hinted at
>>>>> the JVM memory sizes in conf/accumulo-env.sh
>>>>>
>>>>> The monitor may not be working because your jvm/os security profile is
>>>>> different from those we've seen to date.
>>>>> Replace conf/monitor.security.policy with:
>>>>>
>>>>>  grant{
>>>>>   permission java.security.AllPermission;
>>>>> };
>>>>>
>>>>> Or just chase down all the security violations you are getting in your
>>>>> jvm/OS.  A quick look at the monitor logs and they seem ok.
>>>>>
>>>>> Your tserver isn't starting because some other process is listening on the
>>>>> port.  The monitor can't talk to the master because it's waiting for some
>>>>> tablet servers to show up.  The monitor may not display until it talks to
>>>>> the master.  Good news: the logger started up fine.
>>>>>
>>>>> Do this:
>>>>>
>>>>> echo localhost > conf/gc
>>>>>
>>>>> That will get the gc to start up.
>>>>>
>>>>> Then this:
>>>>>
>>>>> # kill everything accumulo related
>>>>> pkill -9 -f accumulo.start
>>>>>
>>>>> # start everything again
>>>>> ./bin/start-all.sh
>>>>>
>>>>> Most of these issues, as well as some example configs for specific memory
>>>>> sizes are fixed in the accumule-1.4.0 release candidate.
>>>>>
>>>>> -Eric
>>>>>
>>>>>
>>>>> On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <[EMAIL PROTECTED]>
>>>>> wrote:
>>>>>>
>>>>>> accumulo init seemed to work. Full disclosure, I am running accumlo in
>>>>>> the MapR M5 vm. I cranked down the memory allocations in
>>>>>> accumulo-env.sh to compensate (maybe too much?). I'm not seeing all of
>>>>>> the process launch. Does the monitor and GC run in their own
>>>>>> processes? Neither the monitoring web page nor accumulo shell are
>>>>>> working. Accumulo shell output below, in addition, ... running
>>>>>> processes, logs, and confs attached.
>>>>>>
>>>>>> Running the whole MapR/Accumulo stack all in a single VM is flirting
>>>>>> with oxymoron. My goal was just to go through the install sequence and
>>>>>> run a few samples. If there is little hope in getting this to work in
>>>>>> a vm, please let me now.
>>>>>>
>>>>>>
>>>>>> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo shell
>>>>>> Enter current password for 'mapr'@'myinstance': *******
>>>>>> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an available
>>>>>> server in the list of servers: [mapr-desktop:9999:9999 (120000)]
>>>>>> 16 08:41:15,974 [shell.Shell] ERROR:
>>>>>> org.apache.accumulo.core.client.AccumuloException:
>>>>>> org.apache.thrift.transport.TTransportException: Failed to connect to
>>>>>> a server
>>>>>>
>>>>>>
>>>>>> Here's what's running:
>>>>>>
>>>>>> mapr     31166  0.2  3.8 708684 78332 ?        Sl   08:08   0:04
>>>>>> /usr/lib/jvm/java-6-sun-1.6.0.26/bin/java -Dapp=master -classpath
>>>>>>
>>>>>> /opt/accumulo-1.3.5-incubating//conf:/opt/accumulo-1.3.5-incubating//src/start/target/classes:/opt/accumulo-1.3.5-incubating//lib/accumulo-start-1.3.5-incubating.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-core-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/commons-jci-fam-1.0.jar:/opt/accumulo-1.3.5-incubating//lib/log4j-1.2.16.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-1.0.4.jar:/opt/accumulo-1.3.5-incubating//lib/commons-logging-api-1.0.4.jar

Todd,

Did you gather any performance numbers?  If so, how did it compare to HDFS?
 And are there any plans to try it on a larger cluster (I'm assuming you
were working with one VM).

Thanks,

--Jason

On Mon, Mar 26, 2012 at 12:11 AM, Todd Stavish <[EMAIL PROTECTED]>wrote:

> Final update, I tested 1.4.0rc5 against MapR v1.2.3. Everything works
> well. The only necessary install steps are the port mapping ones. Very
> clean.
>
> -Todd
>
>
> On Tue, Mar 20, 2012 at 9:39 PM, Todd Stavish <[EMAIL PROTECTED]>
> wrote:
> > Here are the complete instructions for running Accumulo on MapR M5 VM ->
> >
> > http://t.co/RJJ8Ht4B
> >
> > Thanks for all of the help,
> > -Todd
> >
> >
> >
> >
> > On Tue, Mar 20, 2012 at 1:10 PM, Todd Stavish <[EMAIL PROTECTED]>
> wrote:
> >>> I looked at the code and its just simply cast to a
> >>> DistributedFileSystem inorder to check if HDFS is in safe mode.  It
> >>> should do an instanceof operation and then do the case.  Would you
> >>> mind opening a ticket about this?
> >>
> >> Thanks Keith. Done.
> >>
> >> https://issues.apache.org/jira/browse/ACCUMULO-476
> >>
> >>
> >>
> >>>
> >>>>
> >>>>
> >>>> On Fri, Mar 16, 2012 at 9:19 AM, Eric Newton <[EMAIL PROTECTED]>
> wrote:
> >>>>> The monitor and gc do run in their own processes, which is kinda
> hinted at
> >>>>> the JVM memory sizes in conf/accumulo-env.sh
> >>>>>
> >>>>> The monitor may not be working because your jvm/os security profile
> is
> >>>>> different from those we've seen to date.
> >>>>> Replace conf/monitor.security.policy with:
> >>>>>
> >>>>>  grant{
> >>>>>   permission java.security.AllPermission;
> >>>>> };
> >>>>>
> >>>>> Or just chase down all the security violations you are getting in
> your
> >>>>> jvm/OS.  A quick look at the monitor logs and they seem ok.
> >>>>>
> >>>>> Your tserver isn't starting because some other process is listening
> on the
> >>>>> port.  The monitor can't talk to the master because it's waiting for
> some
> >>>>> tablet servers to show up.  The monitor may not display until it
> talks to
> >>>>> the master.  Good news: the logger started up fine.
> >>>>>
> >>>>> Do this:
> >>>>>
> >>>>> echo localhost > conf/gc
> >>>>>
> >>>>> That will get the gc to start up.
> >>>>>
> >>>>> Then this:
> >>>>>
> >>>>> # kill everything accumulo related
> >>>>> pkill -9 -f accumulo.start
> >>>>>
> >>>>> # start everything again
> >>>>> ./bin/start-all.sh
> >>>>>
> >>>>> Most of these issues, as well as some example configs for specific
> memory
> >>>>> sizes are fixed in the accumule-1.4.0 release candidate.
> >>>>>
> >>>>> -Eric
> >>>>>
> >>>>>
> >>>>> On Fri, Mar 16, 2012 at 12:00 PM, Todd Stavish <
> [EMAIL PROTECTED]>
> >>>>> wrote:
> >>>>>>
> >>>>>> accumulo init seemed to work. Full disclosure, I am running accumlo
> in
> >>>>>> the MapR M5 vm. I cranked down the memory allocations in
> >>>>>> accumulo-env.sh to compensate (maybe too much?). I'm not seeing all
> of
> >>>>>> the process launch. Does the monitor and GC run in their own
> >>>>>> processes? Neither the monitoring web page nor accumulo shell are
> >>>>>> working. Accumulo shell output below, in addition, ... running
> >>>>>> processes, logs, and confs attached.
> >>>>>>
> >>>>>> Running the whole MapR/Accumulo stack all in a single VM is flirting
> >>>>>> with oxymoron. My goal was just to go through the install sequence
> and
> >>>>>> run a few samples. If there is little hope in getting this to work
> in
> >>>>>> a vm, please let me now.
> >>>>>>
> >>>>>>
> >>>>>> mapr@mapr-desktop:/opt/accumulo-1.3.5-incubating/bin$ ./accumulo
> shell
> >>>>>> Enter current password for 'mapr'@'myinstance': *******
> >>>>>> 16 08:41:15,971 [impl.ServerClient] WARN : Failed to find an
> available
> >>>>>> server in the list of servers: [mapr-desktop:9999:9999 (120000)]
> >>>>>> 16 08:41:15,974 [shell.Shell] ERROR:
> >>>>>> org.apache.accumulo.core.client.AccumuloException:
> >>>>>> org.apache.thrift.transport.TTransportException: Failed to connect

Jason,

> Did you gather any performance numbers?

A comparison is the end goal. I am in the planning stages and could
use some guidance.

> If so, how did it compare to HDFS? And are there any plans to try it on a larger cluster (I'm assuming you were working with one VM).

Yes, so far, I have deployed MapR\Accumulo to a single VM using
accumulo shell as a smoke test. My plan is to run it in multinode
testing on EC2. I saw a good write-up on accumulodata.com that
describes setting up cloudera to run in EC2.

1. How many nodes should I run to have it be a meaningful test?
2. RC5 seems to have some memory allocation \ config issues. Wait for RC6?
3. Is /test/system/continuous sufficient to test?

The MapR guys have offered to allow me to test on real hardware. So we
could have an EC2 comparison against real servers as well. Also, Mapr
has a Whirr setup. Trying to figure out if I should create one for
accumulo too?

-Todd

On Mon, Mar 26, 2012 at 1:08 PM, Todd Stavish <[EMAIL PROTECTED]> wrote:

> 1. How many nodes should I run to have it be a meaningful test?
>

We test on 1, 10, and ~100 node clusters.
> 2. RC5 seems to have some memory allocation \ config issues. Wait for RC6?
>

RC6 is now available.
> 3. Is /test/system/continuous sufficient to test?
>

Yes.  It will collect performance numbers for you over time, too.
> The MapR guys have offered to allow me to test on real hardware. So we
> could have an EC2 comparison against real servers as well. Also, Mapr
> has a Whirr setup. Trying to figure out if I should create one for
> accumulo too?
That would be great!

-Eric

Todd,

I think a comparison of the performance between MapR and HDFS on 10-20
nodes would be good enough unless you have more nodes available.  If you
could test on real hardware that would be ideal IMO, but EC2 would be fine.
 I look forward to hearing your results.

Thanks,

--Jason

On Mon, Mar 26, 2012 at 2:45 PM, Eric Newton <[EMAIL PROTECTED]> wrote:

>
>
> On Mon, Mar 26, 2012 at 1:08 PM, Todd Stavish <[EMAIL PROTECTED]>wrote:
>
>> 1. How many nodes should I run to have it be a meaningful test?
>>
>
> We test on 1, 10, and ~100 node clusters.
>
>
>> 2. RC5 seems to have some memory allocation \ config issues. Wait for RC6?
>>
>
> RC6 is now available.
>
>
>> 3. Is /test/system/continuous sufficient to test?
>>
>
> Yes.  It will collect performance numbers for you over time, too.
>
>
>> The MapR guys have offered to allow me to test on real hardware. So we
>> could have an EC2 comparison against real servers as well. Also, Mapr
>> has a Whirr setup. Trying to figure out if I should create one for
>> accumulo too?
>
>
> That would be great!
>
> -Eric
>