Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> mapreduce matrix multiplication on hadoop


+
ChWaqas 2011-11-30, 11:23
Copy link to this message
-
Re: mapreduce matrix multiplication on hadoop
The error is that you cannot open /tmp/MatrixMultiply/out/_logs
Does the directory exist?
Do you have proper access rights set?

Joep

On Wed, Nov 30, 2011 at 3:23 AM, ChWaqas <[EMAIL PROTECTED]> wrote:

>
> Hi I am trying to run the matrix multiplication example mentioned(with
> source
> code) on the following link:
>
> http://www.norstad.org/matrix-multiply/index.html
>
> I have hadoop setup in pseudodistributed mode and I configured it using
> this
> tutorial:
>
>
> http://hadoop-tutorial.blogspot.com/2010/11/running-hadoop-in-pseudo-distributed.html?showComment=1321528406255#c3661776111033973764
>
> When I run my jar file then I get the following error:
>
> Identity test
> 11/11/30 10:37:34 INFO input.FileInputFormat: Total input paths to process
> :
> 2
> 11/11/30 10:37:34 INFO mapred.JobClient: Running job: job_201111291041_0010
> 11/11/30 10:37:35 INFO mapred.JobClient:  map 0% reduce 0%
> 11/11/30 10:37:44 INFO mapred.JobClient:  map 100% reduce 0%
> 11/11/30 10:37:56 INFO mapred.JobClient:  map 100% reduce 100%
> 11/11/30 10:37:58 INFO mapred.JobClient: Job complete:
> job_201111291041_0010
> 11/11/30 10:37:58 INFO mapred.JobClient: Counters: 17
> 11/11/30 10:37:58 INFO mapred.JobClient:   Job Counters
> 11/11/30 10:37:58 INFO mapred.JobClient:     Launched reduce tasks=1
> 11/11/30 10:37:58 INFO mapred.JobClient:     Launched map tasks=2
> 11/11/30 10:37:58 INFO mapred.JobClient:     Data-local map tasks=2
> 11/11/30 10:37:58 INFO mapred.JobClient:   FileSystemCounters
> 11/11/30 10:37:58 INFO mapred.JobClient:     FILE_BYTES_READ=114
> 11/11/30 10:37:58 INFO mapred.JobClient:     HDFS_BYTES_READ=248
> 11/11/30 10:37:58 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=298
> 11/11/30 10:37:58 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=124
> 11/11/30 10:37:58 INFO mapred.JobClient:   Map-Reduce Framework
> 11/11/30 10:37:58 INFO mapred.JobClient:     Reduce input groups=2
> 11/11/30 10:37:58 INFO mapred.JobClient:     Combine output records=0
> 11/11/30 10:37:58 INFO mapred.JobClient:     Map input records=4
> 11/11/30 10:37:58 INFO mapred.JobClient:     Reduce shuffle bytes=60
> 11/11/30 10:37:58 INFO mapred.JobClient:     Reduce output records=2
> 11/11/30 10:37:58 INFO mapred.JobClient:     Spilled Records=8
> 11/11/30 10:37:58 INFO mapred.JobClient:     Map output bytes=100
> 11/11/30 10:37:58 INFO mapred.JobClient:     Combine input records=0
> 11/11/30 10:37:58 INFO mapred.JobClient:     Map output records=4
> 11/11/30 10:37:58 INFO mapred.JobClient:     Reduce input records=4
> 11/11/30 10:37:58 INFO input.FileInputFormat: Total input paths to process
> :
> 1
> 11/11/30 10:37:59 INFO mapred.JobClient: Running job: job_201111291041_0011
> 11/11/30 10:38:00 INFO mapred.JobClient:  map 0% reduce 0%
> 11/11/30 10:38:09 INFO mapred.JobClient:  map 100% reduce 0%
> 11/11/30 10:38:21 INFO mapred.JobClient:  map 100% reduce 100%
> 11/11/30 10:38:23 INFO mapred.JobClient: Job complete:
> job_201111291041_0011
> 11/11/30 10:38:23 INFO mapred.JobClient: Counters: 17
> 11/11/30 10:38:23 INFO mapred.JobClient:   Job Counters
> 11/11/30 10:38:23 INFO mapred.JobClient:     Launched reduce tasks=1
> 11/11/30 10:38:23 INFO mapred.JobClient:     Launched map tasks=1
> 11/11/30 10:38:23 INFO mapred.JobClient:     Data-local map tasks=1
> 11/11/30 10:38:23 INFO mapred.JobClient:   FileSystemCounters
> 11/11/30 10:38:23 INFO mapred.JobClient:     FILE_BYTES_READ=34
> 11/11/30 10:38:23 INFO mapred.JobClient:     HDFS_BYTES_READ=124
> 11/11/30 10:38:23 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=100
> 11/11/30 10:38:23 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=124
> 11/11/30 10:38:23 INFO mapred.JobClient:   Map-Reduce Framework
> 11/11/30 10:38:23 INFO mapred.JobClient:     Reduce input groups=2
> 11/11/30 10:38:23 INFO mapred.JobClient:     Combine output records=2
> 11/11/30 10:38:23 INFO mapred.JobClient:     Map input records=2
> 11/11/30 10:38:23 INFO mapred.JobClient:     Reduce shuffle bytes=0
> 11/11/30 10:38:23 INFO mapred.JobClient:     Reduce output records=2
+
Lance Norskog 2011-12-28, 20:46