Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Pregel


Hello,
I don't have a  background in CS, but does MS's Dryad (
http://research.microsoft.com/en-us/projects/Dryad/ ) fit in anywhere
here?
Regards
Saptarshi
On Fri, Jun 26, 2009 at 5:19 AM, Edward J. Yoon<[EMAIL PROTECTED]> wrote:
> According to my understanding, I think the Pregel is in same layer
> with MR, not a MR based language processor.
>
> I think the 'Collective Communication' of BSP seems the core of the
> problem. For example, this BFS problem
> (http://blog.udanax.org/2009/02/breadth-first-search-mapreduce.html)
> can be solved at once w/o MR iterations.
>
> On Fri, Jun 26, 2009 at 3:17 PM, Owen O'Malley<[EMAIL PROTECTED]> wrote:
>>
>> On Jun 25, 2009, at 9:42 PM, Mark Kerzner wrote:
>>
>>> my guess, as good as anybody's, is that Pregel is to large graphs is what
>>> Hadoop is to large datasets.
>>
>> I think it is much more likely a language that allows you to easily define
>> fixed point algorithms.  I would imagine a distributed version of something
>> similar to Michal Young's GenSet.
>> http://portal.acm.org/citation.cfm?doid=586094.586108
>>
>> I've been trying to figure out how to justify working on a project like that
>> for a couple of years, but haven't yet. (I have a background in program
>> static analysis, so I've implemented similar stuff.)
>>
>>> In other words, Pregel is the next natural step
>>> for massively scalable computations after Hadoop.
>>
>> I wonder if it uses map/reduce as a base or not. It would be easier to use
>> map/reduce, but a direct implementation would be more performant. In either
>> case, it is a new hammer. From what I see, it likely won't replace
>> map/reduce, pig, or hive; but rather support a different class of
>> applications much more directly than you can under map/reduce.
>>
>> -- Owen
>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon @ NHN, corp.
> [EMAIL PROTECTED]
> http://blog.udanax.org
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB