Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Bigtop >> mail # user >> Whats a good venue to share bigtop examples ? blog? code? g+ hangout?


Copy link to this message
-
Re: Whats a good venue to share bigtop examples ? blog? code? g+ hangout?
Wiki sounds like a good idea for it. Why don't you post it there and we can
polish/improve it if needed?

On Wed, Sep 04, 2013 at 06:23PM, Jay Vyas wrote:
> Hi folks.  I was considering sharing some ideas / scripts about how we use
> bigtop to test stuff either as code (i.e shell script which invokes various
> maven tasks and copies files) to an examples/ folder in the main repo, or
> else, write a bigtop blog post (i.e. that walks through the script and
> explains the tasks).
>
> Not sure if its worth an "official" code commit, or an official blog post
> or wether I should just dump it on my personal blog and move on ...  But
> let me know ! :)... Here is the basic contents of what the blog post or
> example script will walk through.
>
> 0) Exports environmental variables
>
> 1) builds test-artifacts using standard maven online snapshot mode to get
> dependencies and stuff.
>
> 2) then rebuilds the artifacts in offline mode to overlay our test-artifact
> customizations on top of the work done in the previous step.
>
> 3) cp's some pom.xml files into the test-execution/test-artifacts folders,
> including a custom iTest groovy snippet to run some pig shell tests.
>
> 4) starts flume
>
> 5) runs the test-execution task
>
> Overall, it took a while for me to get it fine tuned to be a good cluster
> test, but now I'm quite happy with it.
>
> I think it could be a useful artifact to anyone that wants to run bigtop
> smoke tests against a non standard hadoop deployment (i.e. anyone like me
> ... :) )
>
> I'm leaning towards sharing this as a blog post since the script is built
> to run against a heavily customized bigtop fork of mine (
> github.com/jayunit100/bigtop) which is quite raw and needs some cleanup....
> but if someone in the bigtop community wants to collaborate, maybe we can
> streamline the example script so that people can use it as a template for
> customizing bigtop smokes to their own needs.
>
> Another option, possibly, is we could have a google hangout in the bigtop
> community where we share the way we use bigtop to do different tasks and
> provide each other with feedback.
>
> --
> Jay Vyas
> http://jayunit100.blogspot.co <http://jayunit100.blogspot.com>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB