Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Bigtop >> mail # user >> New jiras (thanks) :)


+
Jay Vyas 2013-09-25, 13:47
Copy link to this message
-
Re: New jiras (thanks) :)
Hi Jay,

thanks for the kind words. Here's one thing that I wanted to comment on:

On Wed, Sep 25, 2013 at 6:47 AM, Jay Vyas <[EMAIL PROTECTED]> wrote:
> Thanks for all these new jiras focused on end users roman / cons !!(documentation, simplifying hive, jenkins)...
>
> The hive one is sooo true: it's annoying that I need a hive server running to do smoke tests.
> Would like to just smoke test hive ETL without the extra stuff for simpler clusters.
>
> I think in general there should be different types of smokes so that people don't require a whole
> installed infrastructure to run smokes.  Although the vendor distros expect all services running,
> individual users or small startups don't necessarily run all the ecosystem services ,
> they just run the bare Apis, and smoke testing those is still of value on a cluster.

I think this boils down to a few fundamental improvements that we'd have to have
in our test framework to make it flexible:
    * let tests declare (via annotations or somehow else) what
environment they expect
    * have more topologies defined via puppet or similar provisioning frameworks
    * embrace containers Docker.io or similar to not require a full fledged VMs

The above 3 would get us a lot of mileage.

> We do that instead of running the complex hive ETL. For pig we do something's similar ,
> since we want to test basic pig API and not only the new .11 Apis, which are the only one
> supported currently in bigtop .
>
> Anyways.. Thanks for keeping an eye out in this! Jiras like this will make bigtop more
> accessible to the masses !:)

I think its all about more and more folks feeling compelled to contribute small
incremental improvements. Lets hope this will continue!

Thanks,
Roman.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB