Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Bigtop >> mail # user >> Terasort/Teragen in smokes


Copy link to this message
-
Re: Terasort/Teragen in smokes
Thanks Jay, I will try to review it today!

On Thu, Sep 05, 2013 at 01:45PM, Jay Vyas wrote:
> Hi bigtop!
>
> The 1057 patch is now up to date with teragen/terasort benchmarks.
> See https://issues.apache.org/jira/browse/BIGTOP-1057 .   Let me know if
> this looks okay.
>
> Later we can open another ticket for the YCSB benchmarks which will also be
> a great, and simple smoke to add for smoke testing HBase.
>
> On Wed, Aug 28, 2013 at 12:02 AM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>
> > On Tue, Aug 27, 2013 at 10:18PM, Jay Vyas wrote:
> > > hmm ok. now. Thinking about teragen makes me think of benchmarking..
> > >
> > >  In the longer term we could add benchmarking jobs to all the submodules
> > not
> > >  just mapreduce.  For example there are hi bench and ycsb workloads which
> > >  might be usable or pulled in as bigtop components ... Iff of course
> > >  benchmarking is in the cards for bigtop?
> >
> > It indeed is!
> >
> > I think doing tera-gen/sort so it can be parameterized will provide a good
> > basis for future benchmarking (as a bit of reflection: I have did
> > simplistic
> > yet efficient way of benchmarking HDFS and MR a couple years ago, but my
> > employer back then has never let it go into the open. Go figure...)
> >
> > And I have a way of building YCSB against a particular version of Hadoop,
> > so I
> > guess I will have it packaged as a benchmarking test pretty soon.
> >
> > Cos
> >
> > > On Aug 27, 2013, at 7:39 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> > >
> > > > On Tue, Aug 27, 2013 at 4:26 PM, Jay Vyas <[EMAIL PROTECTED]>
> > wrote:
> > > >> Hi guys:
> > > >>
> > > >> I run TeraSort/TeraGen as additions to bigtop in some shell scripts.
> > > >>
> > > >> Any interest in these as an update to TestHadoopExamples in the
> > MapReduce
> > > >> smokes?
> > > >>
> > > >> If so I could patch them in :)
> > > >
> > > > Sure! Sounds like a useful addition.
> > > >
> > > > Thanks,
> > > > Roman.
> >
>
>
>
> --
> Jay Vyas
> http://jayunit100.blogspot.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB