-Re: DBOutputFormat with one reducer
Marcos Ortiz 2011-05-25, 23:22
On 05/25/2011 04:27 PM, Giridhar Addepalli wrote:
> We have MapReduce program which writes data to mysql database using
> Our program has one reducer.
> I understand that all the inserts happen during the close() operation
> of the reducer.
> Is it gauranteed that this operation is atomic ? i.e; what happens if
> the writes fail in the middle of the operation.
> Does it mean that only partial number of rows get in to mysql database?
> What does it take to make the write operation atomic ?
> Any suggestions around our situation( alternative solutions ) are welcome.
Sqoop is done for that kind of tasks.
======= Sqoop is an open-source tool that allows users to extract data from a
relational database into Hadoop for further processing.
This processing can be done with MapReduce programs or other
higher-level tools such as Hive. When the final results of an analytic
pipeline are available, Sqoop can export
these results back to the database for consumption by other clients.
Marcos Luis Ortiz Valmaseda
Software Engineer (Distributed Systems)