| clear query|facets|time |
Search criteria: .
Results from 11 to 20 from
39 (0.182s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Hadoop on physical Machines compared to Amazon Ec2 / virtual machines - Hadoop - [mail # user]
|
|
...Sandeep, How are you guys moving 100 TB into the AWS cloud? Are you using S3 or EBS? If you are using S3, it does not work like HDFS. Although data is replicated (I believe within an a...
|
|
|
Author: Jane Wayne,
2012-06-01, 08:23
|
|
|
Re: how do i view the local file system output of a mapper on cygwin + windows? - Hadoop - [mail # user]
|
|
...i found out what my problem was. apparently, when you iterate over Iterable values, that instance of Type is being used over and over. for example, in my reducer, public void reduce(Ke...
|
|
|
Author: Jane Wayne,
2012-04-06, 00:56
|
|
|
how do i view the local file system output of a mapper on cygwin + windows? - Hadoop - [mail # user]
|
|
...i am currently testing my map reduce job on Windows + Cygwin + Hadoop v0.20.205. for some strange reason, the list of values (i.e. Iterable values) going into the reducer looks all wrong. i ...
|
|
|
Author: Jane Wayne,
2012-04-05, 20:55
|
|
|
Re: how to fine tuning my map reduce job that is generating a lot of intermediate key-value pairs (a lot of I/O operations) - Hadoop - [mail # user]
|
|
...serge, i specify 15 instances, but only 14 end up being data/tasks nodes. 1 instance is reserved as the name node (job tracker). On Wed, Apr 4, 2012 at 1:17 PM, Serge Blazhievsky ...
|
|
|
Author: Jane Wayne,
2012-04-05, 03:15
|
|
|
Re: how to fine tuning my map reduce job that is generating a lot of intermediate key-value pairs (a lot of I/O operations) - Hadoop - [mail # user]
|
|
...i don't have the option of setting the map heap size to 2 GB since my real environment is AWS EMR and the constraints are set. http://hadoop.apache.org/common/docs/r0.20.2/mapred_tutor...
|
|
|
Author: Jane Wayne,
2012-04-04, 03:11
|
|
|
how to fine tuning my map reduce job that is generating a lot of intermediate key-value pairs (a lot of I/O operations) - Hadoop - [mail # user]
|
|
...i have a map reduce job that is generating a lot of intermediate key-value pairs. for example, when i am 1/3 complete with my map phase, i may have generated over 130,000,000 output records ...
|
|
|
Author: Jane Wayne,
2012-04-03, 08:38
|
|
|
Re: how to unit test my RawComparator - Hadoop - [mail # user]
|
|
...chris, 1. thanks, that approach to converting my custom key to byte[] works. 2. on the issue of pass by reference or pass by value, (it's been a while since i've visited this iss...
|
|
|
Author: Jane Wayne,
2012-04-01, 05:32
|
|
|
Re: what is the code for WritableComparator.readVInt and WritableUtils.decodeVIntSize doing? - Hadoop - [mail # user]
|
|
...chris, thanks. i see now. internally, i use String instead of Text and so I use WritableUtils.writeString(...) and not Text.write(...). in the latter method, i see that it calls ...
|
|
|
Author: Jane Wayne,
2012-04-01, 03:19
|
|
|
what is the code for WritableComparator.readVInt and WritableUtils.decodeVIntSize doing? - Hadoop - [mail # user]
|
|
...in tom white's book, Hadoop, The Definitive Guide, in the second edition, on page 99, he shows how to compare the raw bytes of a key with Text fields. he shows an example like the following....
|
|
|
Author: Jane Wayne,
2012-03-31, 04:38
|
|
|
how to unit test my RawComparator - Hadoop - [mail # user]
|
|
...i have a RawComparator that i would like to unit test (using mockito and mrunit testing packages). i want to test the method, public int compare(byte[] b1, int s1, int l1, byte[] b2, i...
|
|
|
Author: Jane Wayne,
2012-03-31, 04:23
|
|
|
|
