Hi all,

I have made some progress in polishing Chukwa 0.5.0.  I think it is
almost ready for public consumption.  Documents have been updated
to reflect the current code base.  I generated a preview of the r0.5.0
document at:

http://people.apache.org/~eyang/chukwa-0.5.0-docs/

Please review and try out the build by running:

mvn clean package site \
-DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
-DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}

If all goes well, we can vote for branching for 0.5.0, then follow by
release votes.
Your input to this matter is important.  Thanks for your attention.

regards,
Eric

I checked the documentation. Looks Good, much clearer and cleaner than
previous stuff. Great work Eric.

My only concern is the first diagram on the Architecture page, which
mention HDFS but doesn't allude to HBase. Is this diagram (in its current
state) still relevant?

On Tue, Dec 6, 2011 at 5:31 AM, Eric Yang <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I have made some progress in polishing Chukwa 0.5.0.  I think it is
> almost ready for public consumption.  Documents have been updated
> to reflect the current code base.  I generated a preview of the r0.5.0
> document at:
>
> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
>
> Please review and try out the build by running:
>
> mvn clean package site \
> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
>
> If all goes well, we can vote for branching for 0.5.0, then follow by
> release votes.
> Your input to this matter is important.  Thanks for your attention.
>
> regards,
> Eric
>

--
Ahmed Fathalla

We can change the diagram from Data Sink to HBase, and remove (5
minutes), and (forever).
It may be more properly describing the architecture.  Any thoughts,
Ari and Jerome?

regards,
Eric

On Mon, Dec 5, 2011 at 8:43 PM, Ahmed Fathalla <[EMAIL PROTECTED]> wrote:
> I checked the documentation. Looks Good, much clearer and cleaner than
> previous stuff. Great work Eric.
>
> My only concern is the first diagram on the Architecture page, which
> mention HDFS but doesn't allude to HBase. Is this diagram (in its current
> state) still relevant?
>
>
>
> On Tue, Dec 6, 2011 at 5:31 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>
>> Hi all,
>>
>> I have made some progress in polishing Chukwa 0.5.0.  I think it is
>> almost ready for public consumption.  Documents have been updated
>> to reflect the current code base.  I generated a preview of the r0.5.0
>> document at:
>>
>> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
>>
>> Please review and try out the build by running:
>>
>> mvn clean package site \
>> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
>> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
>>
>> If all goes well, we can vote for branching for 0.5.0, then follow by
>> release votes.
>> Your input to this matter is important.  Thanks for your attention.
>>
>> regards,
>> Eric
>>
>
>
>
> --
> Ahmed Fathalla

I am unhappy with a build that requires HBase. Is there a way we can
make this optional, both at compile and run time?

--Ari

On Mon, Dec 5, 2011 at 7:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> Hi all,
>
> I have made some progress in polishing Chukwa 0.5.0.  I think it is
> almost ready for public consumption.  Documents have been updated
> to reflect the current code base.  I generated a preview of the r0.5.0
> document at:
>
> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
>
> Please review and try out the build by running:
>
> mvn clean package site \
> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
>
> If all goes well, we can vote for branching for 0.5.0, then follow by
> release votes.
> Your input to this matter is important.  Thanks for your attention.
>
> regards,
> Eric

--
Ari Rabkin [EMAIL PROTECTED]
UC Berkeley Computer Science Department

HBase is optional.  You can build it with "mvn clean package", then
switch the PipelineWriter to SeqFileWriter.  However, the default
behavior is to make everything work as seamless as possible for using
Chukwa as a whole system.

regards,
Eric

On Tue, Dec 6, 2011 at 12:27 PM, Ariel Rabkin <[EMAIL PROTECTED]> wrote:
> I am unhappy with a build that requires HBase. Is there a way we can
> make this optional, both at compile and run time?
>
> --Ari
>
> On Mon, Dec 5, 2011 at 7:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> Hi all,
>>
>> I have made some progress in polishing Chukwa 0.5.0.  I think it is
>> almost ready for public consumption.  Documents have been updated
>> to reflect the current code base.  I generated a preview of the r0.5.0
>> document at:
>>
>> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
>>
>> Please review and try out the build by running:
>>
>> mvn clean package site \
>> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
>> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
>>
>> If all goes well, we can vote for branching for 0.5.0, then follow by
>> release votes.
>> Your input to this matter is important.  Thanks for your attention.
>>
>> regards,
>> Eric
>
>
>
> --
> Ari Rabkin [EMAIL PROTECTED]
> UC Berkeley Computer Science Department

I agree that users should be able to checkout the source and run 'mvn
package' without separately download and installing both Hadoop and HBase.
Can't these unit tests be refactored to not require this by instead
referencing Hadoop and HBase conf dirs that are committed?
On Tue, Dec 6, 2011 at 8:03 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> HBase is optional.  You can build it with "mvn clean package", then
> switch the PipelineWriter to SeqFileWriter.  However, the default
> behavior is to make everything work as seamless as possible for using
> Chukwa as a whole system.
>
> regards,
> Eric
>
> On Tue, Dec 6, 2011 at 12:27 PM, Ariel Rabkin <[EMAIL PROTECTED]> wrote:
> > I am unhappy with a build that requires HBase. Is there a way we can
> > make this optional, both at compile and run time?
> >
> > --Ari
> >
> > On Mon, Dec 5, 2011 at 7:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> >> Hi all,
> >>
> >> I have made some progress in polishing Chukwa 0.5.0.  I think it is
> >> almost ready for public consumption.  Documents have been updated
> >> to reflect the current code base.  I generated a preview of the r0.5.0
> >> document at:
> >>
> >> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
> >>
> >> Please review and try out the build by running:
> >>
> >> mvn clean package site \
> >> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
> >> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
> >>
> >> If all goes well, we can vote for branching for 0.5.0, then follow by
> >> release votes.
> >> Your input to this matter is important.  Thanks for your attention.
> >>
> >> regards,
> >> Eric
> >
> >
> >
> > --
> > Ari Rabkin [EMAIL PROTECTED]
> > UC Berkeley Computer Science Department
>

mvn clean package works fine, maven will automatically download Hadoop
and HBase jar in the build process from maven central.  The
-DHBASE_CONF_DIR and -DHADOOP_CONF_DIR is for developer who wants to
preconfigure the environment variable in chukwa-env.sh.  Without
-DHBASE_CONF_DIR and -DHADOOP_CONF_DIR, chukwa-env.sh is default to
${HBASE_CONF_DIR} and ${HADOOP_CONF_DIR}.  Unit tests are using
minicluster with Hadoop/hbase unit test configuration.  There is no
configuration required to run Hadoop or HBase dependent unit tests.

regards,
Eric

On Fri, Dec 9, 2011 at 5:09 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> I agree that users should be able to checkout the source and run 'mvn
> package' without separately download and installing both Hadoop and HBase.
> Can't these unit tests be refactored to not require this by instead
> referencing Hadoop and HBase conf dirs that are committed?
>
>
> On Tue, Dec 6, 2011 at 8:03 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>
>> HBase is optional.  You can build it with "mvn clean package", then
>> switch the PipelineWriter to SeqFileWriter.  However, the default
>> behavior is to make everything work as seamless as possible for using
>> Chukwa as a whole system.
>>
>> regards,
>> Eric
>>
>> On Tue, Dec 6, 2011 at 12:27 PM, Ariel Rabkin <[EMAIL PROTECTED]> wrote:
>> > I am unhappy with a build that requires HBase. Is there a way we can
>> > make this optional, both at compile and run time?
>> >
>> > --Ari
>> >
>> > On Mon, Dec 5, 2011 at 7:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> >> Hi all,
>> >>
>> >> I have made some progress in polishing Chukwa 0.5.0.  I think it is
>> >> almost ready for public consumption.  Documents have been updated
>> >> to reflect the current code base.  I generated a preview of the r0.5.0
>> >> document at:
>> >>
>> >> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
>> >>
>> >> Please review and try out the build by running:
>> >>
>> >> mvn clean package site \
>> >> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
>> >> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
>> >>
>> >> If all goes well, we can vote for branching for 0.5.0, then follow by
>> >> release votes.
>> >> Your input to this matter is important.  Thanks for your attention.
>> >>
>> >> regards,
>> >> Eric
>> >
>> >
>> >
>> > --
>> > Ari Rabkin [EMAIL PROTECTED]
>> > UC Berkeley Computer Science Department
>>

When I ran ant package the HBase test failed, so I though it was because I
hadn't set HBASE_CONF_DIR. If that's not required, then we have a test
problem:

$ less
target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.txt
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.625 sec
<<< FAILURE!
warning(junit.framework.TestSuite$1)  Time elapsed: 0.005 sec  <<< FAILURE!
junit.framework.AssertionFailedError: Exception in constructor: testWriters
(junit.framework.AssertionFailedError
        at junit.framework.Assert.fail(Assert.java:48)
        at
org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:75)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
        ...

On Fri, Dec 9, 2011 at 9:16 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> mvn clean package works fine, maven will automatically download Hadoop
> and HBase jar in the build process from maven central.  The
> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR is for developer who wants to
> preconfigure the environment variable in chukwa-env.sh.  Without
> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR, chukwa-env.sh is default to
> ${HBASE_CONF_DIR} and ${HADOOP_CONF_DIR}.  Unit tests are using
> minicluster with Hadoop/hbase unit test configuration.  There is no
> configuration required to run Hadoop or HBase dependent unit tests.
>
> regards,
> Eric
>
> On Fri, Dec 9, 2011 at 5:09 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> > I agree that users should be able to checkout the source and run 'mvn
> > package' without separately download and installing both Hadoop and
> HBase.
> > Can't these unit tests be refactored to not require this by instead
> > referencing Hadoop and HBase conf dirs that are committed?
> >
> >
> > On Tue, Dec 6, 2011 at 8:03 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> >
> >> HBase is optional.  You can build it with "mvn clean package", then
> >> switch the PipelineWriter to SeqFileWriter.  However, the default
> >> behavior is to make everything work as seamless as possible for using
> >> Chukwa as a whole system.
> >>
> >> regards,
> >> Eric
> >>
> >> On Tue, Dec 6, 2011 at 12:27 PM, Ariel Rabkin <[EMAIL PROTECTED]>
> wrote:
> >> > I am unhappy with a build that requires HBase. Is there a way we can
> >> > make this optional, both at compile and run time?
> >> >
> >> > --Ari
> >> >
> >> > On Mon, Dec 5, 2011 at 7:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> >> >> Hi all,
> >> >>
> >> >> I have made some progress in polishing Chukwa 0.5.0.  I think it is
> >> >> almost ready for public consumption.  Documents have been updated
> >> >> to reflect the current code base.  I generated a preview of the
> r0.5.0
> >> >> document at:
> >> >>
> >> >> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
> >> >>
> >> >> Please review and try out the build by running:
> >> >>
> >> >> mvn clean package site \
> >> >> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
> >> >> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
> >> >>
> >> >> If all goes well, we can vote for branching for 0.5.0, then follow by
> >> >> release votes.
> >> >> Your input to this matter is important.  Thanks for your attention.
> >> >>
> >> >> regards,
> >> >> Eric
> >> >
> >> >
> >> >
> >> > --
> >> > Ari Rabkin [EMAIL PROTECTED]
> >> > UC Berkeley Computer Science Department
> >>
>

TestHBaseWriter works on my side without setting HBASE_CONF_DIR.  In
the exception, it has printed additional stacktrace to stdout.  Could
you paste the output in
target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter-output.txt?

regards,
Eric

On Fri, Dec 9, 2011 at 11:11 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> When I ran ant package the HBase test failed, so I though it was because I
> hadn't set HBASE_CONF_DIR. If that's not required, then we have a test
> problem:
>
> $ less
> target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.txt
> Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.625 sec
> <<< FAILURE!
> warning(junit.framework.TestSuite$1)  Time elapsed: 0.005 sec  <<< FAILURE!
> junit.framework.AssertionFailedError: Exception in constructor: testWriters
> (junit.framework.AssertionFailedError
>         at junit.framework.Assert.fail(Assert.java:48)
>         at
> org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:75)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>         ...
>
>
> On Fri, Dec 9, 2011 at 9:16 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>
>> mvn clean package works fine, maven will automatically download Hadoop
>> and HBase jar in the build process from maven central.  The
>> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR is for developer who wants to
>> preconfigure the environment variable in chukwa-env.sh.  Without
>> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR, chukwa-env.sh is default to
>> ${HBASE_CONF_DIR} and ${HADOOP_CONF_DIR}.  Unit tests are using
>> minicluster with Hadoop/hbase unit test configuration.  There is no
>> configuration required to run Hadoop or HBase dependent unit tests.
>>
>> regards,
>> Eric
>>
>> On Fri, Dec 9, 2011 at 5:09 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
>> > I agree that users should be able to checkout the source and run 'mvn
>> > package' without separately download and installing both Hadoop and
>> > HBase.
>> > Can't these unit tests be refactored to not require this by instead
>> > referencing Hadoop and HBase conf dirs that are committed?
>> >
>> >
>> > On Tue, Dec 6, 2011 at 8:03 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> >
>> >> HBase is optional.  You can build it with "mvn clean package", then
>> >> switch the PipelineWriter to SeqFileWriter.  However, the default
>> >> behavior is to make everything work as seamless as possible for using
>> >> Chukwa as a whole system.
>> >>
>> >> regards,
>> >> Eric
>> >>
>> >> On Tue, Dec 6, 2011 at 12:27 PM, Ariel Rabkin <[EMAIL PROTECTED]>
>> >> wrote:
>> >> > I am unhappy with a build that requires HBase. Is there a way we can
>> >> > make this optional, both at compile and run time?
>> >> >
>> >> > --Ari
>> >> >
>> >> > On Mon, Dec 5, 2011 at 7:31 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> >> >> Hi all,
>> >> >>
>> >> >> I have made some progress in polishing Chukwa 0.5.0.  I think it is
>> >> >> almost ready for public consumption.  Documents have been updated
>> >> >> to reflect the current code base.  I generated a preview of the
>> >> >> r0.5.0
>> >> >> document at:
>> >> >>
>> >> >> http://people.apache.org/~eyang/chukwa-0.5.0-docs/
>> >> >>
>> >> >> Please review and try out the build by running:
>> >> >>
>> >> >> mvn clean package site \
>> >> >> -DHADOOP_CONF_DIR=/path/to/${HADOOP_CONF_DIR} \
>> >> >> -DHBASE_CONF_DIR=/path/to/${HBASE_CONF_DIR}
>> >> >>
>> >> >> If all goes well, we can vote for branching for 0.5.0, then follow
>> >> >> by
>> >> >> release votes.
>> >> >> Your input to this matter is important.  Thanks for your attention.
>> >> >>
>> >> >> regards,

Starting DataNode 0 with dfs.data.dir:
/Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data1,/Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data2

java.lang.NullPointerException
        at
org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)

        at
org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)

        at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:223)

        at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:345)

        at
org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:308)

        at
org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:66)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)

        at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)

        at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:513)

        at junit.framework.TestSuite.createTest(TestSuite.java:61)

        at junit.framework.TestSuite.addTestMethod(TestSuite.java:294)

        at
junit.framework.TestSuite.addTestsFromTestCase(TestSuite.java:150)

        at junit.framework.TestSuite.<init>(TestSuite.java:129)
On Fri, Dec 9, 2011 at 11:34 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> TestHBaseWriter works on my side without setting HBASE_CONF_DIR.  In
> the exception, it has printed additional stacktrace to stdout.  Could
> you paste the output in
>
> target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter-output.txt?
>
> regards,
> Eric
>
> On Fri, Dec 9, 2011 at 11:11 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> > When I ran ant package the HBase test failed, so I though it was because
> I
> > hadn't set HBASE_CONF_DIR. If that's not required, then we have a test
> > problem:
> >
> > $ less
> >
> target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.txt
> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.625 sec
> > <<< FAILURE!
> > warning(junit.framework.TestSuite$1)  Time elapsed: 0.005 sec  <<<
> FAILURE!
> > junit.framework.AssertionFailedError: Exception in constructor:
> testWriters
> > (junit.framework.AssertionFailedError
> >         at junit.framework.Assert.fail(Assert.java:48)
> >         at
> >
> org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:75)
> >         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> >         at
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >         at
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >         at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >         ...
> >
> >
> > On Fri, Dec 9, 2011 at 9:16 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> >>
> >> mvn clean package works fine, maven will automatically download Hadoop
> >> and HBase jar in the build process from maven central.  The
> >> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR is for developer who wants to
> >> preconfigure the environment variable in chukwa-env.sh.  Without
> >> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR, chukwa-env.sh is default to
> >> ${HBASE_CONF_DIR} and ${HADOOP_CONF_DIR}.  Unit tests are using
> >> minicluster with Hadoop/hbase unit test configuration.  There is no
> >> configuration required to run Hadoop or HBase dependent unit tests.
> >>
> >> regards,
> >> Eric
> >>
> >> On Fri, Dec 9, 2011 at 5:09 PM, Bill Graham <[EMAIL PROTECTED]>
> wrote:
> >> > I agree that users should be able to checkout the source and run 'mvn

This error is generated when MiniDFSCluster can not resolve the local
ip address or hostname.
Make sure the test machine has network setup properly.  Hope this helps.

regards,
Eric

On Sat, Dec 10, 2011 at 7:04 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> Starting DataNode 0 with dfs.data.dir:
> /Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data1,/Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data2
>
> java.lang.NullPointerException
>
>
>         at
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
>
>         at
> org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
>
>         at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:223)
>
>         at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:345)
>
>         at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:308)
>
>         at
> org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:66)
>
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>
>         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>
>         at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>
>         at junit.framework.TestSuite.createTest(TestSuite.java:61)
>
>         at junit.framework.TestSuite.addTestMethod(TestSuite.java:294)
>
>         at
> junit.framework.TestSuite.addTestsFromTestCase(TestSuite.java:150)
>
>         at junit.framework.TestSuite.<init>(TestSuite.java:129)
>
>
> On Fri, Dec 9, 2011 at 11:34 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>
>> TestHBaseWriter works on my side without setting HBASE_CONF_DIR.  In
>> the exception, it has printed additional stacktrace to stdout.  Could
>> you paste the output in
>>
>> target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter-output.txt?
>>
>> regards,
>> Eric
>>
>> On Fri, Dec 9, 2011 at 11:11 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
>> > When I ran ant package the HBase test failed, so I though it was because
>> > I
>> > hadn't set HBASE_CONF_DIR. If that's not required, then we have a test
>> > problem:
>> >
>> > $ less
>> >
>> > target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.txt
>> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.625
>> > sec
>> > <<< FAILURE!
>> > warning(junit.framework.TestSuite$1)  Time elapsed: 0.005 sec  <<<
>> > FAILURE!
>> > junit.framework.AssertionFailedError: Exception in constructor:
>> > testWriters
>> > (junit.framework.AssertionFailedError
>> >         at junit.framework.Assert.fail(Assert.java:48)
>> >         at
>> >
>> > org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:75)
>> >         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> > Method)
>> >         at
>> >
>> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>> >         at
>> >
>> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>> >         at
>> > java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> >         ...
>> >
>> >
>> > On Fri, Dec 9, 2011 at 9:16 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> >>
>> >> mvn clean package works fine, maven will automatically download Hadoop
>> >> and HBase jar in the build process from maven central.  The
>> >> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR is for developer who wants to
>> >> preconfigure the environment variable in chukwa-env.sh.  Without
>> >> -DHBASE_CONF_DIR and -DHADOOP_CONF_DIR, chukwa-env.sh is default to

+1

The tests passed on my home machine. Still not sure why it didn't on my
other one.

On Sat, Dec 10, 2011 at 7:26 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> This error is generated when MiniDFSCluster can not resolve the local
> ip address or hostname.
> Make sure the test machine has network setup properly.  Hope this helps.
>
> regards,
> Eric
>
> On Sat, Dec 10, 2011 at 7:04 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> > Starting DataNode 0 with dfs.data.dir:
> >
> /Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data1,/Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data2
> >
> > java.lang.NullPointerException
> >
> >
> >         at
> >
> org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
> >
> >         at
> > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
> >
> >         at
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:223)
> >
> >         at
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:345)
> >
> >         at
> >
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:308)
> >
> >         at
> >
> org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:66)
> >
> >         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> > Method)
> >
> >         at
> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >
> >         at
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >
> >         at
> java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >
> >         at junit.framework.TestSuite.createTest(TestSuite.java:61)
> >
> >         at junit.framework.TestSuite.addTestMethod(TestSuite.java:294)
> >
> >         at
> > junit.framework.TestSuite.addTestsFromTestCase(TestSuite.java:150)
> >
> >         at junit.framework.TestSuite.<init>(TestSuite.java:129)
> >
> >
> > On Fri, Dec 9, 2011 at 11:34 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> >>
> >> TestHBaseWriter works on my side without setting HBASE_CONF_DIR.  In
> >> the exception, it has printed additional stacktrace to stdout.  Could
> >> you paste the output in
> >>
> >>
> target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter-output.txt?
> >>
> >> regards,
> >> Eric
> >>
> >> On Fri, Dec 9, 2011 at 11:11 PM, Bill Graham <[EMAIL PROTECTED]>
> wrote:
> >> > When I ran ant package the HBase test failed, so I though it was
> because
> >> > I
> >> > hadn't set HBASE_CONF_DIR. If that's not required, then we have a test
> >> > problem:
> >> >
> >> > $ less
> >> >
> >> >
> target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.txt
> >> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.625
> >> > sec
> >> > <<< FAILURE!
> >> > warning(junit.framework.TestSuite$1)  Time elapsed: 0.005 sec  <<<
> >> > FAILURE!
> >> > junit.framework.AssertionFailedError: Exception in constructor:
> >> > testWriters
> >> > (junit.framework.AssertionFailedError
> >> >         at junit.framework.Assert.fail(Assert.java:48)
> >> >         at
> >> >
> >> >
> org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:75)
> >> >         at
> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> >> > Method)
> >> >         at
> >> >
> >> >
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> >> >         at
> >> >
> >> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> >> >         at
> >> > java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> >> >         ...
> >> >
> >> >
> >> > On Fri, Dec 9, 2011 at 9:16 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

+1

On Mon, Dec 12, 2011 at 10:41 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
> +1
>
> The tests passed on my home machine. Still not sure why it didn't on my
> other one.
>
>
> On Sat, Dec 10, 2011 at 7:26 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>>
>> This error is generated when MiniDFSCluster can not resolve the local
>> ip address or hostname.
>> Make sure the test machine has network setup properly.  Hope this helps.
>>
>> regards,
>> Eric
>>
>> On Sat, Dec 10, 2011 at 7:04 PM, Bill Graham <[EMAIL PROTECTED]> wrote:
>> > Starting DataNode 0 with dfs.data.dir:
>> >
>> > /Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data1,/Users/billg/ws/svn/chukwa/target/test/classes/target/test-data/f104e7b0-0999-4234-9df8-290189ec3674/dfs/data/data2
>> >
>> > java.lang.NullPointerException
>> >
>> >
>> >         at
>> >
>> > org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:422)
>> >
>> >         at
>> > org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
>> >
>> >         at
>> >
>> > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:223)
>> >
>> >         at
>> >
>> > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:345)
>> >
>> >         at
>> >
>> > org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:308)
>> >
>> >         at
>> >
>> > org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:66)
>> >
>> >         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> > Method)
>> >
>> >         at
>> >
>> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>> >
>> >         at
>> >
>> > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>> >
>> >         at
>> > java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>> >
>> >         at junit.framework.TestSuite.createTest(TestSuite.java:61)
>> >
>> >         at junit.framework.TestSuite.addTestMethod(TestSuite.java:294)
>> >
>> >         at
>> > junit.framework.TestSuite.addTestsFromTestCase(TestSuite.java:150)
>> >
>> >         at junit.framework.TestSuite.<init>(TestSuite.java:129)
>> >
>> >
>> > On Fri, Dec 9, 2011 at 11:34 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
>> >>
>> >> TestHBaseWriter works on my side without setting HBASE_CONF_DIR.  In
>> >> the exception, it has printed additional stacktrace to stdout.  Could
>> >> you paste the output in
>> >>
>> >>
>> >> target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter-output.txt?
>> >>
>> >> regards,
>> >> Eric
>> >>
>> >> On Fri, Dec 9, 2011 at 11:11 PM, Bill Graham <[EMAIL PROTECTED]>
>> >> wrote:
>> >> > When I ran ant package the HBase test failed, so I though it was
>> >> > because
>> >> > I
>> >> > hadn't set HBASE_CONF_DIR. If that's not required, then we have a
>> >> > test
>> >> > problem:
>> >> >
>> >> > $ less
>> >> >
>> >> >
>> >> > target/test-reports/org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.txt
>> >> > Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.625
>> >> > sec
>> >> > <<< FAILURE!
>> >> > warning(junit.framework.TestSuite$1)  Time elapsed: 0.005 sec  <<<
>> >> > FAILURE!
>> >> > junit.framework.AssertionFailedError: Exception in constructor:
>> >> > testWriters
>> >> > (junit.framework.AssertionFailedError
>> >> >         at junit.framework.Assert.fail(Assert.java:48)
>> >> >         at
>> >> >
>> >> >
>> >> > org.apache.hadoop.chukwa.datacollection.writer.TestHBaseWriter.<init>(TestHBaseWriter.java:75)
>> >> >         at
>> >> > sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
>> >> > Method)
>> >> >         at
>> >> >
>> >> >
>> >> > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>> >> >         at
>> >> >
>> >> >
>>