Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # general >> Data World Record Falls as Computer Scientists Break Terabyte Sort Barrier


Copy link to this message
-
Re: Data World Record Falls as Computer Scientists Break Terabyte Sort Barrier
Maybe this would give us a little more incentive to resolve HDFS-347

Cheers

On Mon, Aug 2, 2010 at 1:50 PM, Abhishek Verma <[EMAIL PROTECTED]>wrote:

> It shows how further behind Hadoop is in terms of performance. Are there
> people working on finding the bottlenecks and making it more efficient? Are
> there any JIRA issues related to this?
>
> -Abhishek.
>
> On Mon, Aug 2, 2010 at 11:47 AM, Arun C Murthy <[EMAIL PROTECTED]> wrote:
>
> > The UCSD results are very impressive, especially given their hardware
> > budget.
> >
> > I may be wrong, but I'm pretty sure there were no Hadoop based entries
> this
> > year - I know we at Yahoo! didn't enter.
> >
> > Couple of points:
> > # The Indy category is a benchmark to sort fixed length records, not a
> > _general_ sort benchmark i.e. Daytona.
> > # Our _best_ result missed the deadline by a whisker last year, but we
> > eventually did 100Tb sort in 95 mins and a 1000TB (1PB) in 975 mins
> (16.25
> > hrs) - which worked out to be just over 1.0 TB/min, which was nearly
> twice
> > as fast as the record attributed to us. (
> >
> http://developer.yahoo.net/blogs/hadoop/2009/05/hadoop_sorts_a_petabyte_in_162.html
> > )
> >
> > Arun
> >
> >
> > On Aug 2, 2010, at 10:34 AM, Abhishek Verma wrote:
> >
> >  Hi Maxim,
> >>
> >> Hadoop was not involved. You can find more details here :
> >> http://sortbenchmark.org/tritonsort_2010_May_15.pdf
> >> and all the records and their information here :
> >> http://sortbenchmark.org/
> >>
> >> <http://sortbenchmark.org/>-Abhishek.
> >>
> >> On Mon, Aug 2, 2010 at 9:52 AM, Maxim Veksler <[EMAIL PROTECTED]>
> wrote:
> >>
> >>  Hi,
> >>>
> >>> Anyone knows if Hadoop is involved? And if so what is the configuration
> >>> for
> >>> such cluster?
> >>>
> >>> http://ucsdnews.ucsd.edu/newsrel/science/07-27DataWorld.asp
> >>>
> >>>
> >>> Thank you,
> >>> Maxim.
> >>>
> >>>
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB