Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Issues in emitting 2D double array to Reducer.


Copy link to this message
-
Re: Issues in emitting 2D double array to Reducer.
unmesha sreeveni 2013-10-29, 04:05
Am i doing right?

public class MatrixWritable implements WritableComparable<MatrixWritable>{

/**
 * @param args
 */
private double[][] value1;
private double[][] value2;
 public MatrixWritable(){
 int row1 = value1.length;
int col1 = value1[0].length;
this.value1 = new double[row1][col1];
 int row2 = value2.length;
int col2 = value2[0].length;
this.value2 = new double[row2][col2];
}
 @Override
public void readFields(DataInput in) throws IOException {
// TODO Auto-generated method stub
  }

@Override
public void write(DataOutput out) throws IOException {
// TODO Auto-generated method stub
 }

 @Override
public int compareTo(MatrixWritable o) {
// TODO Auto-generated method stub
return 0;
}

}

but how to set the readfields?
On Tue, Oct 29, 2013 at 8:45 AM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:

> http://mapreduce-tutorial.blogspot.in/2011/04/custom-data-types.html
> I have gone through this link.What i understood is for emitting two 2D
> double arrays as key and value from mapper and read those two arrays as key
> and value from reducer ..i need to create a class which implements a
> writable ... am i right?
>
> And how to override those methods in writable.
> These r all new to me. But i am picking up.
>
>
> On Mon, Oct 28, 2013 at 8:52 PM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:
>
>> thanx for ur replies Mirko Kämpf and bertrand. i hv read in many of the
>> document for  the creation of custom datatypes. i have nt tried with
>> advanced datatypes.I will Try and see.
>>
>> Before tat...this wont fulfill my need right???
>>
>> *public static class DoubleArrayWritable extends TwoDArrayWritable {
>>         public DoubleArrayWritable() {
>>             super(DoubleWritable.class);
>>         }
>>     }
>> *
>>
>> * DoubleArrayWritable EArray = new DoubleArrayWritable();
>>             DoubleWritable[][] Edata = new DoubleWritable[Erow][Ecol];
>>             for (int k = 0; k < Erow; k++) {
>>                 for(int j=0;j< Ecol;j++){
>>                     Edata[k][j] = new DoubleWritable(E[k][j]);
>>                 }
>>             }
>>             EArray.set(Edata);
>>
>>             DoubleArrayWritable EDArray = new DoubleArrayWritable();
>>             DoubleWritable[][] EDdata = new DoubleWritable[EtransDerow][EtransDecol];
>>             for (int k1 = 0; k1 < EtransDerow; k1++) {
>>                 for(int j1=0;j1< EtransDecol;j1++){
>>                 EDdata[k1][j1] = new DoubleWritable(ED[k1][j1]);
>>
>>             }
>>             }
>>             EDArray.set(EDdata);***
>>
>>     *context**.write(EArray, EDArray);*
>>
>>
>> The 2D array i need to emit is Edata and EDArray.I am changing the double array to *DoubleArrayWritable *
>> ,but y this dont work?
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Oct 28, 2013 at 5:22 PM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:
>>
>>> Use @Override to specify that you indeed override the good methods.
>>>
>>> Bertrand
>>>
>>>
>>> On Mon, Oct 28, 2013 at 11:30 AM, Mirko Kämpf <[EMAIL PROTECTED]>wrote:
>>>
>>>> Hi,
>>>>
>>>> you should create a custom data type, which contains two of the
>>>> DoubleArrayWritable instances. This custom data type idea is also
>>>> explained here:
>>>> http://my.safaribooksonline.com/book/-/9781849517287/4dot-developing-complex-hadoop-mapreduce-applications/ch04s03_html
>>>>
>>>> Good luck.
>>>> Mirko
>>>>
>>>>
>>>>
>>>> 2013/10/28 unmesha sreeveni <[EMAIL PROTECTED]>
>>>>
>>>>>
>>>>> http://stackoverflow.com/questions/19602692/java-lang-classcastexception-in-hadoop-mapreduce
>>>>>
>>>>> I tried out, but it is not working.
>>>>>
>>>>> I was able to emit 1 2D array..but When i cross checked once more, i
>>>>> am not able to print the matrix(2D array) in reducer.
>>>>>
>>>>> Can anyone suggest me a gud way to emit 2 Matrices from mapper.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Thanks & Regards*
>>>>> *
>>>>> *
>>>>> Unmesha Sreeveni U.B*
>>>>> *
>>>>> *Junior Developer
>>>>> *
>>>>> *Amrita Center For Cyber Security
>>>>> *
>>>>> *
>>>>> Amritapuri.

*Thanks & Regards*
*
*
Unmesha Sreeveni U.B*
*
*Junior Developer
*
*Amrita Center For Cyber Security
*
*
Amritapuri.

www.amrita.edu/cyber/
*