Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Hadoop example code (preferred Pagerank and Shortest Path)


Copy link to this message
-
Re: Hadoop example code (preferred Pagerank and Shortest Path)
Ravi Prakash 2012-05-07, 21:54
Shuai,

I'm afraid I don't know if there are open-sourced implementations of those
two algorithms, but there are a bunch of example programs in the
hadoop-mapreduce-examples*.jar that gets built / distributed.

Valid program names are:
  aggregatewordcount: An Aggregate based map/reduce program that counts the
words in the input files.
  aggregatewordhist: An Aggregate based map/reduce program that computes
the histogram of the words in the input files.
  dbcount: An example job that count the pageview counts from a database.
  grep: A map/reduce program that counts the matches of a regex in the
input.
  join: A job that effects a join over sorted, equally partitioned datasets
  multifilewc: A job that counts words from several files.
  pentomino: A map/reduce tile laying program to find solutions to
pentomino problems.
  pi: A map/reduce program that estimates Pi using monte-carlo method.
  randomtextwriter: A map/reduce program that writes 10GB of random textual
data per node.
  randomwriter: A map/reduce program that writes 10GB of random data per
node.
  secondarysort: An example defining a secondary sort to the reduce.
  sleep: A job that sleeps at each map and reduce task.
  sort: A map/reduce program that sorts the data written by the random
writer.
  sudoku: A sudoku solver.
  teragen: Generate data for the terasort
  terasort: Run the terasort
  teravalidate: Checking results of terasort
  wordcount: A map/reduce program that counts the words in the input files.

HTH
Ravi

On Mon, May 7, 2012 at 3:56 PM, Shuai Zheng <[EMAIL PROTECTED]>wrote:

> Hi all,
>
> I am student working on Hadoop 0.20.203.0.
>
> I want to monitor the system resource utilization of pagerank and/or
> shortest path hadoop program. I only need to monitor the system, so I did
> not write any hadoop programs.
>
> I want to ask for some example codes (with many iterations) and dataset
> from you. Algorithms such as:
> 1. Pagerank
> 2. Shortest path
>
> (This is not necessary) it is better that the program can output log info
> for iteration start time and stop time.
>
> If you know any public example code, or you have your own code, and would
> like to share with me (in case any related publication using your code in
> the future, your contribution will be acknowledged.), please send me a
> copy.
>
> Thank you and best regards,
>
> Shuai
>