Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Joins in Sqoop

Copy link to this message
Re: Joins in Sqoop
Large joins are better performed on the database side and stored in temporary table (CREATE TABLE temp_tbl AS SELECT ...) that can be subsequently imported by Sqoop without creating large temp files.


On Thu, Jan 16, 2014 at 09:20:04PM +0800, Sambit Tripathy (RBEI/PJ-NBS) wrote:
> Hi,
> I have written query which has 5 Join clauses and I am passing this query in Sqoop import.
> Problem: This produces a large temp file in the MySQL server temp directory and throws back an error saying No Space left on the device. Yes this can be fixed by increasing the size of the temp directory in the MySQL server, but what if you actually don't have any space left on MySQL server. Are there any workarounds for this? I mean something like a batch import which does not create a big temp file in the server.
> Regards,
> Sambit.