Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # dev - Revisit Pig Philosophy?


Copy link to this message
-
Revisit Pig Philosophy?
Santhosh Srinivasan 2009-09-18, 22:54
Pig Developers,

I looked at the Pig philosophy page as it serves as a guideline for
accepting changes to Pig. Is it time to revisit the overall philosophy?

Reference: http://hadoop.apache.org/pig/philosophy.html

Some items of interest:

1. SQL semantics and Pig

With the recent addition of SQL on top of Pig, we are making changes to
accommodate SQL semantics. Should this be part of Pig's philosophy?

2. Local mode and other parallel frameworks

<snip>
Pigs Live Anywhere

Pig is intended to be a language for parallel data processing. It is not
tied to one particular parallel framework. It has been implemented first
on hadoop, but we do not intend that to be only on hadoop.
</snip>

Are we still holding onto this? What about local mode? Local mode is not
being treated on equal footing with that of Hadoop for practical
reasons. However, users expect things that work on local mode to work
without any hitches on Hadoop.

Are we still designing the system assuming that Pig will be stacked on
top of other parallel frameworks?

Thanks,
Santhosh