Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Issues in emitting 2D double array to Reducer.


Copy link to this message
-
Re: Issues in emitting 2D double array to Reducer.
Am i doing right?

public class MatrixWritable implements WritableComparable<MatrixWritable>{

/**
 * @param args
 */
private double[][] value1;
private double[][] value2;
 public MatrixWritable(){
 int row1 = value1.length;
int col1 = value1[0].length;
this.value1 = new double[row1][col1];
 int row2 = value2.length;
int col2 = value2[0].length;
this.value2 = new double[row2][col2];
}
 @Override
public void readFields(DataInput in) throws IOException {
// TODO Auto-generated method stub
  }

@Override
public void write(DataOutput out) throws IOException {
// TODO Auto-generated method stub
 }

 @Override
public int compareTo(MatrixWritable o) {
// TODO Auto-generated method stub
return 0;
}

}

but how to set the readfields?
On Tue, Oct 29, 2013 at 8:45 AM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:

> http://mapreduce-tutorial.blogspot.in/2011/04/custom-data-types.html
> I have gone through this link.What i understood is for emitting two 2D
> double arrays as key and value from mapper and read those two arrays as key
> and value from reducer ..i need to create a class which implements a
> writable ... am i right?
>
> And how to override those methods in writable.
> These r all new to me. But i am picking up.
>
>
> On Mon, Oct 28, 2013 at 8:52 PM, unmesha sreeveni <[EMAIL PROTECTED]>wrote:
>
>> thanx for ur replies Mirko Kämpf and bertrand. i hv read in many of the
>> document for  the creation of custom datatypes. i have nt tried with
>> advanced datatypes.I will Try and see.
>>
>> Before tat...this wont fulfill my need right???
>>
>> *public static class DoubleArrayWritable extends TwoDArrayWritable {
>>         public DoubleArrayWritable() {
>>             super(DoubleWritable.class);
>>         }
>>     }
>> *
>>
>> * DoubleArrayWritable EArray = new DoubleArrayWritable();
>>             DoubleWritable[][] Edata = new DoubleWritable[Erow][Ecol];
>>             for (int k = 0; k < Erow; k++) {
>>                 for(int j=0;j< Ecol;j++){
>>                     Edata[k][j] = new DoubleWritable(E[k][j]);
>>                 }
>>             }
>>             EArray.set(Edata);
>>
>>             DoubleArrayWritable EDArray = new DoubleArrayWritable();
>>             DoubleWritable[][] EDdata = new DoubleWritable[EtransDerow][EtransDecol];
>>             for (int k1 = 0; k1 < EtransDerow; k1++) {
>>                 for(int j1=0;j1< EtransDecol;j1++){
>>                 EDdata[k1][j1] = new DoubleWritable(ED[k1][j1]);
>>
>>             }
>>             }
>>             EDArray.set(EDdata);***
>>
>>     *context**.write(EArray, EDArray);*
>>
>>
>> The 2D array i need to emit is Edata and EDArray.I am changing the double array to *DoubleArrayWritable *
>> ,but y this dont work?
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Oct 28, 2013 at 5:22 PM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:
>>
>>> Use @Override to specify that you indeed override the good methods.
>>>
>>> Bertrand
>>>
>>>
>>> On Mon, Oct 28, 2013 at 11:30 AM, Mirko Kämpf <[EMAIL PROTECTED]>wrote:
>>>
>>>> Hi,
>>>>
>>>> you should create a custom data type, which contains two of the
>>>> DoubleArrayWritable instances. This custom data type idea is also
>>>> explained here:
>>>> http://my.safaribooksonline.com/book/-/9781849517287/4dot-developing-complex-hadoop-mapreduce-applications/ch04s03_html
>>>>
>>>> Good luck.
>>>> Mirko
>>>>
>>>>
>>>>
>>>> 2013/10/28 unmesha sreeveni <[EMAIL PROTECTED]>
>>>>
>>>>>
>>>>> http://stackoverflow.com/questions/19602692/java-lang-classcastexception-in-hadoop-mapreduce
>>>>>
>>>>> I tried out, but it is not working.
>>>>>
>>>>> I was able to emit 1 2D array..but When i cross checked once more, i
>>>>> am not able to print the matrix(2D array) in reducer.
>>>>>
>>>>> Can anyone suggest me a gud way to emit 2 Matrices from mapper.
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> *Thanks & Regards*
>>>>> *
>>>>> *
>>>>> Unmesha Sreeveni U.B*
>>>>> *
>>>>> *Junior Developer
>>>>> *
>>>>> *Amrita Center For Cyber Security
>>>>> *
>>>>> *
>>>>> Amritapuri.

*Thanks & Regards*
*
*
Unmesha Sreeveni U.B*
*
*Junior Developer
*
*Amrita Center For Cyber Security
*
*
Amritapuri.

www.amrita.edu/cyber/
*
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB