Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - storing intermediate results ?

Copy link to this message
storing intermediate results ?
Vincent BARAT 2009-10-07, 14:54

I'm new to PIG, and I have a bunch of statements that process the
same input, which is actually the result of a JOIN between two very
big data set (millions of entries).

I wonder if it is better (faster) to save the result of this JOIN
into an Hadoop file and then to LOAD it, instead of just relying on
PIG optimizations ?

Thank a lot for your help.
Ashutosh Chauhan 2009-10-07, 16:33
zaki rahaman 2009-10-07, 20:08
Thejas Nair 2009-10-07, 20:16
Vincent BARAT 2009-10-08, 13:33
Alan Gates 2009-10-12, 18:50
Vincent BARAT 2009-10-08, 09:43