| clear query|facets|time |
Search criteria: .
Results from 1 to 10 from
112 (0.187s).
|
|
|
Loading phrases to help you
refine your search...
|
|
Re: Setting number of mappers according to number of TextInput lines - Hadoop - [mail # user]
|
|
...How did you try it? I had no problem with NLineInputFormat. It just works exactly as expected. Shi...
|
|
|
Author: Shi Yu,
2012-06-16, 22:33
|
|
|
Re: Hadoop on physical Machines compared to Amazon Ec2 / virtual machines - Hadoop - [mail # user]
|
|
...We once calculated the cost of using EC2 to train our machine learning model (assuming we did everything in one shot, which is almost impossible) using EM algorithm. The cost for...
|
|
|
Author: Shi Yu,
2012-05-31, 21:43
|
|
|
Re: Random Sample in Map/Reduce - Hadoop - [mail # user]
|
|
...To answer my own question. I applied a non-repeatable random number generator in the mapper. At mapper setup stage I generate a pre-defined number of random numbers, then I...
|
|
|
Author: Shi Yu,
2012-05-15, 02:23
|
|
|
Random Sample in Map/Reduce - Hadoop - [mail # user]
|
|
...Hi, Before I raise this question I searched relevant topics. There are suggestions online: "Mappers: Output all qualifying values, each with a random integer key. &nb...
|
|
|
Author: Shi Yu,
2012-05-15, 01:33
|
|
|
Re: transferring between HDFS which reside in different subnet - Hadoop - [mail # user]
|
|
...It seems in your case HDFS2 could access HDFS, so you should be able to transfer HDFS data to HDFS2. If you want to cross-transfer, you don't need to do distcp on cluster n...
|
|
|
Author: Shi Yu,
2012-05-11, 14:14
|
|
|
Re: How to maintain record boundaries - Hadoop - [mail # user]
|
|
...here are some quick code for you (based on Tom's book). You could overwrite the TextInputFormat isSplitable method to avoid splitting, which is pretty important and useful when p...
|
|
|
Author: Shi Yu,
2012-05-11, 14:08
|
|
|
Re: freeze a mapreduce job - Hadoop - [mail # user]
|
|
...Is there any risk to suppress a job too long in FS? I guess there are some parameters to control the waiting time of a job (such as timeout ,etc.), for exampl...
|
|
|
Author: Shi Yu,
2012-05-11, 14:03
|
|
|
Re: transferring between HDFS which reside in different subnet - Hadoop - [mail # user]
|
|
...If you could cross-access HDFS from both name nodes, then it should be transferable using /distcp /command. Shi * * On 5/11/2012 8:45 AM, Arindam Choudhury wrote: ...
|
|
|
Author: Shi Yu,
2012-05-11, 13:57
|
|
|
Re: SQL analysis - Hadoop - [mail # user]
|
|
...If the analysis you mention is to create "view" of multiple tables. Once your data is sorted by the keys in HDFS. You could try Map Side join or Reducer Side join in Hadoop to ge...
|
|
|
Author: Shi Yu,
2012-05-10, 15:13
|
|
|
RE: SQL analysis - Hadoop - [mail # user]
|
|
...Flume might be suitable for your case. https://cwiki.apache.org/FLUME/ Shi...
|
|
|
Author: Shi Yu,
2012-05-10, 13:28
|
|
|
|
