modemide 2011-09-01, 19:47
-Re: MultipleOutputs - Create multiple files during output
Stan Rosenberg 2011-09-02, 02:24
You could create a custom HashPartitioner so that all key,value pairs
denoting the actions of the same user end up in the same reducer; then you
only one output file per reducer. Btw, how large are the output files? make
sure you don't end up creating
a lot of small files, i.e., << 64MB.
On Thu, Sep 1, 2011 at 3:47 PM, modemide <[EMAIL PROTECTED]> wrote:
> Hi all,
> I was wondering if anyone was familiar with this class. I want to
> create multiple output files during my reduce.
> My input files will consist of
> My goal is to create files with the following format
> File Contents:
> I.e. This will store all the actions of one person for any given month
> in one file.
> I just don't know how I will decide the file name at run time. Can anyone