Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # dev - Join implementations


+
Prashant Kommireddi 2012-11-13, 23:20
+
Ashutosh Chauhan 2012-11-14, 03:01
+
Prashant Kommireddi 2012-11-14, 07:28
Copy link to this message
-
Re: Join implementations
Jonathan Coveney 2012-11-14, 02:47
A key class that aids in understanding how the physical layer works is the
LogToPhyTranslationVisitor. You can look at the visitor for the LOJoin
logical operator and see what it does for different join types (FRJoin
being the easier).

The code around plan generation is IMHO some of the most difficult to
understand in Pig, so if you can get your head around how to take the
logical plan and stitch together an optimized join (ie skewed join) then it
will be much easier to debug errors in the physical plan, and potentially
make new operators.
2012/11/13 Prashant Kommireddi <[EMAIL PROTECTED]>

> Hi All,
>
> What would be a good starting point for me to understand the various Join
> implementations in Pig codebase?
>
> Thanks,
> Prashant
>