Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> storing intermediate results ?

Copy link to this message
storing intermediate results ?

I'm new to PIG, and I have a bunch of statements that process the
same input, which is actually the result of a JOIN between two very
big data set (millions of entries).

I wonder if it is better (faster) to save the result of this JOIN
into an Hadoop file and then to LOAD it, instead of just relying on
PIG optimizations ?

Thank a lot for your help.