Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Matrix multiplication in Hadoop


Copy link to this message
-
Re: Matrix multiplication in Hadoop
Hey Mike
          In mahout one place where   matrix multiplication is used is in  Collaborative Filtering distributed implementation. The recommendations here are generated by the multiplication of a cooccurence matrix with a user vector. This user vector is treated as a single column matrix and then the matrix multiplication takes place in there.

Regards
Bejoy K S

-----Original Message-----
From: Mike Spreitzer <[EMAIL PROTECTED]>
Date: Fri, 18 Nov 2011 14:52:05
To: <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: RE: Matrix multiplication in Hadoop

Well, this mismatch may tell me something interesting about Hadoop. Matrix
multiplication has a lot of inherent parallelism, so from very crude
considerations it is not obvious that there should be a mismatch.  Why is
matrix multiplication ill-suited for Hadoop?

BTW, I looked into the Mahout documentation some, and did not find matrix
multiplication there.  It might be hidden inside one of the advertised
algorithms; I looked at the documentation for a few, but did not notice
mention of MM.

Thanks,
Mike

From:   Michael Segel <[EMAIL PROTECTED]>
To:     <[EMAIL PROTECTED]>
Date:   11/18/2011 01:49 PM
Subject:        RE: Matrix multiplication in Hadoop
Ok Mike,

First I admire that you are studying Hadoop.

To answer your question... not well.

Might I suggest that if you want to learn Hadoop, you try and find a
problem which can easily be broken in to a series of parallel tasks where
there is minimal communication requirements between each task?

No offense, but if I could make a parallel... what you're asking is akin
to taking a normalized relational model and trying to run it as is in
HBase.
Yes it can be done. But not the best use of resources.

> To: [EMAIL PROTECTED]
> CC: [EMAIL PROTECTED]
> Subject: Re: Matrix multiplication in Hadoop
> From: [EMAIL PROTECTED]
> Date: Fri, 18 Nov 2011 12:39:00 -0500
>
> That's also an interesting question, but right now I am studying Hadoop
> and want to know how well dense MM can be done in Hadoop.
>
> Thanks,
> Mike
>
>
>
> From:   Michel Segel <[EMAIL PROTECTED]>
> To:     "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Date:   11/18/2011 12:34 PM
> Subject:        Re: Matrix multiplication in Hadoop
>
>
>
> Is Hadoop the best tool for doing large matrix math.
> Sure you can do it, but, aren't there better tools for these types of
> problems?
>
>
> Sent from a remote device. Please excuse any typos...
>
> Mike Segel
>