Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> What is the correct way to get a string back from a mapper or reducer


Copy link to this message
-
Re: What is the correct way to get a string back from a mapper or reducer
The stackoverflow question doesn't add any useful information.

Like I said you can emit the string inside a record. Or if you really want
to handle lots of complexity, write it yourself within a file or a
datastore from the reducer. But you will then have to consider performance
issues and be able to handle to lifecycle of the task, its potential
multiple attempts and the global lifecyle of the job itself. So it's not
necessary obvious, it would depend on the context.

The concept of "global variable" in distributed computing should be well
understood. By essence, its not possible to have a distributed,
always-available, always-consistent variable (see CAP).

Bertrand Dechoux
On Thu, Jul 3, 2014 at 7:51 AM, Chris MacKenzie <
[EMAIL PROTECTED]> wrote: