Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - multiple input splits from single file


+
sharat attupurath 2012-06-10, 15:02
+
Harsh J 2012-06-10, 15:06
Copy link to this message
-
Re: multiple input splits from single file
Karthik Kambatla 2012-06-11, 06:54
Hi Sharat

A couple of questions/comments:

   1. Is your input graph complete?
   2. If it is not complete, it might make sense to use adjacency lists of
   graph nodes as the input to each map function. (Multiple adjacency lists
   for the map task)
   3. Even if it is complete, using the adjacency lists - or a partition of
   the edges as input for each map task might help.

Karthik

On Sun, Jun 10, 2012 at 8:02 AM, sharat attupurath <[EMAIL PROTECTED]>wrote:

>  Hi,
>
> We are trying to solve the travelling salesman problem using hadoop. our
> input files contain just a single line that has the euclidean coordinates
> of the cities. we need to pass this single line to each mapper who will
> then process that. How can we do this so that we can achieve parallelism in
> a hadoop cluster. Is there any way to generate multiple input splits from
> the single input file.
>
> Thanks
>
> Sharat
>