Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Managing Large Pig Scripts

Copy link to this message
Re: Managing Large Pig Scripts

You might consider breaking it into one or more reusable macros. What version of pig are you using?

For complicated scripts, especially if you didn't write them, you might want to take a look at lipstick, https://github.com/Netflix/Lipstick
It allows you to visualize the dag and clearly shows what logical operators map to map-reduce jobs. It could be a starting point for managing complexity at least.

On Mar 5, 2014, at 7:33 AM, Christopher Petrino <[EMAIL PROTECTED]> wrote: