Dear all,

I am a Ph.D. student in Computer Science and have 4-year Java
programming experience focusing on Java Web development.
In the candidate projects in PIG, I am interested in PIG-2586 (A better
plan/data flow visualizer) and PIG-2599 (Mavenize Pig).

In my on-going research project, I am in charge of (1). web user
interface development and (2). build system. Now I am working on adding
hadoop capability to the project. The main reason I am interested in the
PIG project is that I can make a contribution to the PIG community based
on my previous experience,  and learn from the participant in GSoC this
year and benefit my on-going research project at the same time.

(1). User interface development
I have used several graphic libraries to visualize semantic data and our
own data set, e.g., Jung, graphviz, BIRT, and several plot plugins in
jquery.
Therefore, I am interested in working on a new tool for PIG visualizer.
After looking through the bug issue, I have several questions:
     (i) As both swing and javascript are mentioned, is this project a
web or standalone application?
     (ii) As ruby-graphviz is included, Is ruby required for this project?

(2). Build system
The code base of my research project is 40K loc and the build script was
written in Ant. Part of my duty is to convert the ant build script to
maven and maintain the build script. Therefore, Mavenize Pig is of
interest to me too. The build.xml in PIG project is more complicated
than the one I worked before. It includes ant, maven and ivy. Do we need
to use maven to do all the tasks and get rid of all the dependency on
ant, maven and ivy?

  Best regards
  Shasha(Amy) Liu

See comments inline.

On Sat, Mar 17, 2012 at 6:52 AM, grassonsand <[EMAIL PROTECTED]> wrote:
> Dear all,
>
> I am a Ph.D. student in Computer Science and have 4-year Java programming
> experience focusing on Java Web development.
> In the candidate projects in PIG, I am interested in PIG-2586 (A better
> plan/data flow visualizer) and PIG-2599 (Mavenize Pig).
>
> In my on-going research project, I am in charge of (1). web user interface
> development and (2). build system. Now I am working on adding hadoop
> capability to the project. The main reason I am interested in the PIG
> project is that I can make a contribution to the PIG community based on my
> previous experience,  and learn from the participant in GSoC this year and
> benefit my on-going research project at the same time.
>
> (1). User interface development
> I have used several graphic libraries to visualize semantic data and our own
> data set, e.g., Jung, graphviz, BIRT, and several plot plugins in jquery.
> Therefore, I am interested in working on a new tool for PIG visualizer.
> After looking through the bug issue, I have several questions:
>    (i) As both swing and javascript are mentioned, is this project a web or
> standalone application?
>    (ii) As ruby-graphviz is included, Is ruby required for this project?

I envision two visualize components in Pig. One is a lightweight
visualizer invoked by Grunt, which should be fast and concise, and
integrated into explain command. The other is a standalone composer
similar to PigPen, which should be much powerful. PIG-2586 is intended
to track the first, but Russel's comment is talking about the second.
Both are acceptable as a GSoC project. I leave it to Russel.

>
> (2). Build system
> The code base of my research project is 40K loc and the build script was
> written in Ant. Part of my duty is to convert the ant build script to maven
> and maintain the build script. Therefore, Mavenize Pig is of interest to me
> too. The build.xml in PIG project is more complicated than the one I worked
> before. It includes ant, maven and ivy. Do we need to use maven to do all
> the tasks and get rid of all the dependency on ant, maven and ivy?

Yes

>
>  Best regards
>  Shasha(Amy) Liu

Hi Daniel,

Thanks a lot for the reply.
I installed the latest Pig and read through the book of "programming in
pig".
I manged to use "-dot -out filename" to produce three graphs in dot file
format.

Based on the existing dot file, my next question is what is the requirement
regarding a better visualizer?
Are we going to generate a picture (e.g., .png) for different plans
(logical plan, physical plan, map reduce plan), or provide a web interface
to visualize these graphs of plans?

Best regards,
--
Shasha(Amy) Liu

On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> See comments inline.
>
> On Sat, Mar 17, 2012 at 6:52 AM, grassonsand <[EMAIL PROTECTED]>
> wrote:
> > Dear all,
> >
> > I am a Ph.D. student in Computer Science and have 4-year Java programming
> > experience focusing on Java Web development.
> > In the candidate projects in PIG, I am interested in PIG-2586 (A better
> > plan/data flow visualizer) and PIG-2599 (Mavenize Pig).
> >
> > In my on-going research project, I am in charge of (1). web user
> interface
> > development and (2). build system. Now I am working on adding hadoop
> > capability to the project. The main reason I am interested in the PIG
> > project is that I can make a contribution to the PIG community based on
> my
> > previous experience,  and learn from the participant in GSoC this year
> and
> > benefit my on-going research project at the same time.
> >
> > (1). User interface development
> > I have used several graphic libraries to visualize semantic data and our
> own
> > data set, e.g., Jung, graphviz, BIRT, and several plot plugins in jquery.
> > Therefore, I am interested in working on a new tool for PIG visualizer.
> > After looking through the bug issue, I have several questions:
> >    (i) As both swing and javascript are mentioned, is this project a web
> or
> > standalone application?
> >    (ii) As ruby-graphviz is included, Is ruby required for this project?
>
> I envision two visualize components in Pig. One is a lightweight
> visualizer invoked by Grunt, which should be fast and concise, and
> integrated into explain command. The other is a standalone composer
> similar to PigPen, which should be much powerful. PIG-2586 is intended
> to track the first, but Russel's comment is talking about the second.
> Both are acceptable as a GSoC project. I leave it to Russel.
>
> >
> > (2). Build system
> > The code base of my research project is 40K loc and the build script was
> > written in Ant. Part of my duty is to convert the ant build script to
> maven
> > and maintain the build script. Therefore, Mavenize Pig is of interest to
> me
> > too. The build.xml in PIG project is more complicated than the one I
> worked
> > before. It includes ant, maven and ivy. Do we need to use maven to do all
> > the tasks and get rid of all the dependency on ant, maven and ivy?
>
> Yes
>
> >
> >  Best regards
> >  Shasha(Amy) Liu
>

I suggest you create a simple, minimal web application that visualizes a
pig script file each time a url with the script filename is loaded.

For instance, the process to use the tool might go like this:

1) Run pigvisualizer.(pl/py/rb) locally, at the start of your pig work
session
2) Create a new pig script at /my/dif/filename.pig
3) Open http://localhost:4567/pigviz/my/dir/filename.pig in a web browser
4) See a javascript-based visualization of your pig script
5) Reload this web page each time you want to see a new visualization OR
have to page try to reload the file periodically

There are several sources of data:

1) Start a pig session, via grunt,PigServer or HCatalog, and use
ILLUSTRATE/EXPLAIN.  An old example of doing this is available at
https://github.com/rjurney/Cloud-Stenography
2) Use the explain or -dot commands from pig command line. In looking at
the dot output, the graph is not as helpful as I had thought :(
3) Use the PigPen code to get ILLUSTRATE data for visualization

The ideal situation is that you get the data plan via EXPLAIN, and sample
data via ILLUSTRATE, and combine them to produce an even better version of
figure 2 in the paper
http://infolab.stanford.edu/~olston/publications/sigmod09.pdf

[image: Inline image 1]

As to the presentation of the data in an interface, I suggest you AVOID
eclipse and the UI code to PigPen, as there is little utility in having
this visualization there.  Not all Pig users use Eclipse, and there is
little utility in editing scripts in the diagrams.  There is great utility
in visualizing, understanding and debugging this way, but not so much in
editing.

On the other hand, anyone can edit Pig in their favorite tool and view
their pig graph in a simple web application on their localhost by directing
a web browser at it.  This is why a simple, small web application seems
best. You can use ruby/sinatra or python/bottle/flask or perl/catalyst to
make a simple web app.  Check out sigma.js for graph visualization:
http://sigmajs.org/examples.html or http://neyric.github.com/wireit/ for
something more fully featured.

Perhaps the best plan is to fix ILLUSTRATE (see
http://wiki.apache.org/pig/ExampleGenerator and talk to the guys at
mortardata.com who have a patch for this), and edit the PigPen code to
remove the Eclipse dependencies and have it output simple JSON for a web
application to consume.  It could write to a file, or you could create a
simple web service that publishes JSON for the current pig session.

Once we have JSON of ILLUSTRATE... getting a web visualization is easy.  I
can help, I've done it before in Cloud Stenography by parsing data in
Grunt.  Which you could do, btw.  Old Perl code is available on github (see
above link).

Interested in thoughts of others.

On Fri, Mar 23, 2012 at 11:21 PM, Shasha Liu <[EMAIL PROTECTED]> wrote:

> Hi Daniel,
>
> Thanks a lot for the reply.
> I installed the latest Pig and read through the book of "programming in
> pig".
> I manged to use "-dot -out filename" to produce three graphs in dot file
> format.
>
> Based on the existing dot file, my next question is what is the
> requirement regarding a better visualizer?
> Are we going to generate a picture (e.g., .png) for different plans
> (logical plan, physical plan, map reduce plan), or provide a web interface
> to visualize these graphs of plans?
>
> Best regards,
> --
> Shasha(Amy) Liu
>
>
> On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>
>> See comments inline.
>>
>> On Sat, Mar 17, 2012 at 6:52 AM, grassonsand <[EMAIL PROTECTED]>
>> wrote:
>> > Dear all,
>> >
>> > I am a Ph.D. student in Computer Science and have 4-year Java
>> programming
>> > experience focusing on Java Web development.
>> > In the candidate projects in PIG, I am interested in PIG-2586 (A better
>> > plan/data flow visualizer) and PIG-2599 (Mavenize Pig).
>> >
>> > In my on-going research project, I am in charge of (1). web user
>> interface
>> > development and (2). build system. Now I am working on adding hadoop
Russell Jurney twitter.com/rjurney [EMAIL PROTECTED] datasyndrome.com

Hi Russell,

Based on the email discussions, I wrote my proposal of this pig visualizer
project and submit it  onto google-melange. Please take a look at it at
your convenience, and it would also appreciated a lot if further
feedback/comments could be provided.

Thank you very much.
Best,

On Sun, Mar 25, 2012 at 9:25 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:

> I suggest you create a simple, minimal web application that visualizes a
> pig script file each time a url with the script filename is loaded.
>
> For instance, the process to use the tool might go like this:
>
> 1) Run pigvisualizer.(pl/py/rb) locally, at the start of your pig work
> session
> 2) Create a new pig script at /my/dif/filename.pig
> 3) Open http://localhost:4567/pigviz/my/dir/filename.pig in a web browser
> 4) See a javascript-based visualization of your pig script
> 5) Reload this web page each time you want to see a new visualization OR
> have to page try to reload the file periodically
>
> There are several sources of data:
>
> 1) Start a pig session, via grunt,PigServer or HCatalog, and use
> ILLUSTRATE/EXPLAIN.  An old example of doing this is available at
> https://github.com/rjurney/Cloud-Stenography
> 2) Use the explain or -dot commands from pig command line. In looking at
> the dot output, the graph is not as helpful as I had thought :(
> 3) Use the PigPen code to get ILLUSTRATE data for visualization
>
> The ideal situation is that you get the data plan via EXPLAIN, and sample
> data via ILLUSTRATE, and combine them to produce an even better version of
> figure 2 in the paper
> http://infolab.stanford.edu/~olston/publications/sigmod09.pdf
>
> [image: Inline image 1]
>
> As to the presentation of the data in an interface, I suggest you AVOID
> eclipse and the UI code to PigPen, as there is little utility in having
> this visualization there.  Not all Pig users use Eclipse, and there is
> little utility in editing scripts in the diagrams.  There is great utility
> in visualizing, understanding and debugging this way, but not so much in
> editing.
>
> On the other hand, anyone can edit Pig in their favorite tool and view
> their pig graph in a simple web application on their localhost by directing
> a web browser at it.  This is why a simple, small web application seems
> best. You can use ruby/sinatra or python/bottle/flask or perl/catalyst to
> make a simple web app.  Check out sigma.js for graph visualization:
> http://sigmajs.org/examples.html or http://neyric.github.com/wireit/ for
> something more fully featured.
>
> Perhaps the best plan is to fix ILLUSTRATE (see
> http://wiki.apache.org/pig/ExampleGenerator and talk to the guys at
> mortardata.com who have a patch for this), and edit the PigPen code to
> remove the Eclipse dependencies and have it output simple JSON for a web
> application to consume.  It could write to a file, or you could create a
> simple web service that publishes JSON for the current pig session.
>
> Once we have JSON of ILLUSTRATE... getting a web visualization is easy.  I
> can help, I've done it before in Cloud Stenography by parsing data in
> Grunt.  Which you could do, btw.  Old Perl code is available on github (see
> above link).
>
> Interested in thoughts of others.
>
> On Fri, Mar 23, 2012 at 11:21 PM, Shasha Liu <[EMAIL PROTECTED]>wrote:
>
>> Hi Daniel,
>>
>> Thanks a lot for the reply.
>> I installed the latest Pig and read through the book of "programming in
>> pig".
>> I manged to use "-dot -out filename" to produce three graphs in dot file
>> format.
>>
>> Based on the existing dot file, my next question is what is the
>> requirement regarding a better visualizer?
>> Are we going to generate a picture (e.g., .png) for different plans
>> (logical plan, physical plan, map reduce plan), or provide a web interface
>> to visualize these graphs of plans?
>>
>> Best regards,
>>  --
>> Shasha(Amy) Liu
>>
>>
>> On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <[EMAIL PROTECTED]>wrote:
>>
>>> See comments inline.

Shasha(Amy) Liu

That's great, I'll review it now!

Russell Jurney http://datasyndrome.com

On Apr 6, 2012, at 7:03 AM, Shasha Liu <[EMAIL PROTECTED]> wrote:

> Hi Russell,
>
> Based on the email discussions, I wrote my proposal of this pig visualizer project and submit it  onto google-melange. Please take a look at it at your convenience, and it would also appreciated a lot if further feedback/comments could be provided.
>
> Thank you very much.
> Best,
>
> On Sun, Mar 25, 2012 at 9:25 PM, Russell Jurney <[EMAIL PROTECTED]> wrote:
> I suggest you create a simple, minimal web application that visualizes a pig script file each time a url with the script filename is loaded.  
>
> For instance, the process to use the tool might go like this:
>
> 1) Run pigvisualizer.(pl/py/rb) locally, at the start of your pig work session
> 2) Create a new pig script at /my/dif/filename.pig
> 3) Open http://localhost:4567/pigviz/my/dir/filename.pig in a web browser
> 4) See a javascript-based visualization of your pig script
> 5) Reload this web page each time you want to see a new visualization OR have to page try to reload the file periodically
>
> There are several sources of data:
>
> 1) Start a pig session, via grunt,PigServer or HCatalog, and use ILLUSTRATE/EXPLAIN.  An old example of doing this is available at https://github.com/rjurney/Cloud-Stenography
> 2) Use the explain or -dot commands from pig command line. In looking at the dot output, the graph is not as helpful as I had thought :(
> 3) Use the PigPen code to get ILLUSTRATE data for visualization
>
> The ideal situation is that you get the data plan via EXPLAIN, and sample data via ILLUSTRATE, and combine them to produce an even better version of figure 2 in the paper http://infolab.stanford.edu/~olston/publications/sigmod09.pdf
>
> <image.png>
>
> As to the presentation of the data in an interface, I suggest you AVOID eclipse and the UI code to PigPen, as there is little utility in having this visualization there.  Not all Pig users use Eclipse, and there is little utility in editing scripts in the diagrams.  There is great utility in visualizing, understanding and debugging this way, but not so much in editing.
>
> On the other hand, anyone can edit Pig in their favorite tool and view their pig graph in a simple web application on their localhost by directing a web browser at it.  This is why a simple, small web application seems best. You can use ruby/sinatra or python/bottle/flask or perl/catalyst to make a simple web app.  Check out sigma.js for graph visualization: http://sigmajs.org/examples.html or http://neyric.github.com/wireit/ for something more fully featured.
>
> Perhaps the best plan is to fix ILLUSTRATE (see http://wiki.apache.org/pig/ExampleGenerator and talk to the guys at mortardata.com who have a patch for this), and edit the PigPen code to remove the Eclipse dependencies and have it output simple JSON for a web application to consume.  It could write to a file, or you could create a simple web service that publishes JSON for the current pig session.
>
> Once we have JSON of ILLUSTRATE... getting a web visualization is easy.  I can help, I've done it before in Cloud Stenography by parsing data in Grunt.  Which you could do, btw.  Old Perl code is available on github (see above link).
>
> Interested in thoughts of others.
>
> On Fri, Mar 23, 2012 at 11:21 PM, Shasha Liu <[EMAIL PROTECTED]> wrote:
> Hi Daniel,
>
> Thanks a lot for the reply.
> I installed the latest Pig and read through the book of "programming in pig".
> I manged to use "-dot -out filename" to produce three graphs in dot file format.
>
> Based on the existing dot file, my next question is what is the requirement regarding a better visualizer?
> Are we going to generate a picture (e.g., .png) for different plans (logical plan, physical plan, map reduce plan), or provide a web interface to visualize these graphs of plans?
>
> Best regards,
> --
> Shasha(Amy) Liu
>
>
> On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <[EMAIL PROTECTED]> wrote:

I can't see the proposal, as I'm not an approved mentor.  I'd suggest
working with [EMAIL PROTECTED] if possible. You might focus on
visualization of ILLUSTRATE while he focuses on the editor?
On Fri, Mar 23, 2012 at 11:21 PM, Shasha Liu <[EMAIL PROTECTED]> wrote:

> Hi Daniel,
>
> Thanks a lot for the reply.
> I installed the latest Pig and read through the book of "programming in
> pig".
> I manged to use "-dot -out filename" to produce three graphs in dot file
> format.
>
> Based on the existing dot file, my next question is what is the
> requirement regarding a better visualizer?
> Are we going to generate a picture (e.g., .png) for different plans
> (logical plan, physical plan, map reduce plan), or provide a web interface
> to visualize these graphs of plans?
>
> Best regards,
> --
> Shasha(Amy) Liu
>
>
> On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <[EMAIL PROTECTED]> wrote:
>
>> See comments inline.
>>
>> On Sat, Mar 17, 2012 at 6:52 AM, grassonsand <[EMAIL PROTECTED]>
>> wrote:
>> > Dear all,
>> >
>> > I am a Ph.D. student in Computer Science and have 4-year Java
>> programming
>> > experience focusing on Java Web development.
>> > In the candidate projects in PIG, I am interested in PIG-2586 (A better
>> > plan/data flow visualizer) and PIG-2599 (Mavenize Pig).
>> >
>> > In my on-going research project, I am in charge of (1). web user
>> interface
>> > development and (2). build system. Now I am working on adding hadoop
>> > capability to the project. The main reason I am interested in the PIG
>> > project is that I can make a contribution to the PIG community based on
>> my
>> > previous experience,  and learn from the participant in GSoC this year
>> and
>> > benefit my on-going research project at the same time.
>> >
>> > (1). User interface development
>> > I have used several graphic libraries to visualize semantic data and
>> our own
>> > data set, e.g., Jung, graphviz, BIRT, and several plot plugins in
>> jquery.
>> > Therefore, I am interested in working on a new tool for PIG visualizer.
>> > After looking through the bug issue, I have several questions:
>> >    (i) As both swing and javascript are mentioned, is this project a
>> web or
>> > standalone application?
>> >    (ii) As ruby-graphviz is included, Is ruby required for this project?
>>
>> I envision two visualize components in Pig. One is a lightweight
>> visualizer invoked by Grunt, which should be fast and concise, and
>> integrated into explain command. The other is a standalone composer
>> similar to PigPen, which should be much powerful. PIG-2586 is intended
>> to track the first, but Russel's comment is talking about the second.
>> Both are acceptable as a GSoC project. I leave it to Russel.
>>
>> >
>> > (2). Build system
>> > The code base of my research project is 40K loc and the build script was
>> > written in Ant. Part of my duty is to convert the ant build script to
>> maven
>> > and maintain the build script. Therefore, Mavenize Pig is of interest
>> to me
>> > too. The build.xml in PIG project is more complicated than the one I
>> worked
>> > before. It includes ant, maven and ivy. Do we need to use maven to do
>> all
>> > the tasks and get rid of all the dependency on ant, maven and ivy?
>>
>> Yes
>>
>> >
>> >  Best regards
>> >  Shasha(Amy) Liu
>>
>
>
>
--
Russell Jurney twitter.com/rjurney [EMAIL PROTECTED] datasyndrome.com

Hi Russell,

My proposal is available through this link:
http://www.cs.uga.edu/~shasha/gsoc12/pig/application_pig_visualizer.html .

Look forward to your further comments, thank you.

Best,

On Fri, Apr 6, 2012 at 1:37 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:

> I can't see the proposal, as I'm not an approved mentor.  I'd suggest
> working with [EMAIL PROTECTED] if possible. You might focus on
> visualization of ILLUSTRATE while he focuses on the editor?
> On Fri, Mar 23, 2012 at 11:21 PM, Shasha Liu <[EMAIL PROTECTED]>wrote:
>
>> Hi Daniel,
>>
>> Thanks a lot for the reply.
>> I installed the latest Pig and read through the book of "programming in
>> pig".
>> I manged to use "-dot -out filename" to produce three graphs in dot file
>> format.
>>
>> Based on the existing dot file, my next question is what is the
>> requirement regarding a better visualizer?
>> Are we going to generate a picture (e.g., .png) for different plans
>> (logical plan, physical plan, map reduce plan), or provide a web interface
>> to visualize these graphs of plans?
>>
>> Best regards,
>>  --
>> Shasha(Amy) Liu
>>
>>
>> On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <[EMAIL PROTECTED]>wrote:
>>
>>> See comments inline.
>>>
>>> On Sat, Mar 17, 2012 at 6:52 AM, grassonsand <[EMAIL PROTECTED]>
>>> wrote:
>>> > Dear all,
>>> >
>>> > I am a Ph.D. student in Computer Science and have 4-year Java
>>> programming
>>> > experience focusing on Java Web development.
>>> > In the candidate projects in PIG, I am interested in PIG-2586 (A better
>>> > plan/data flow visualizer) and PIG-2599 (Mavenize Pig).
>>> >
>>> > In my on-going research project, I am in charge of (1). web user
>>> interface
>>> > development and (2). build system. Now I am working on adding hadoop
>>> > capability to the project. The main reason I am interested in the PIG
>>> > project is that I can make a contribution to the PIG community based
>>> on my
>>> > previous experience,  and learn from the participant in GSoC this year
>>> and
>>> > benefit my on-going research project at the same time.
>>> >
>>> > (1). User interface development
>>> > I have used several graphic libraries to visualize semantic data and
>>> our own
>>> > data set, e.g., Jung, graphviz, BIRT, and several plot plugins in
>>> jquery.
>>> > Therefore, I am interested in working on a new tool for PIG visualizer.
>>> > After looking through the bug issue, I have several questions:
>>> >    (i) As both swing and javascript are mentioned, is this project a
>>> web or
>>> > standalone application?
>>> >    (ii) As ruby-graphviz is included, Is ruby required for this
>>> project?
>>>
>>> I envision two visualize components in Pig. One is a lightweight
>>> visualizer invoked by Grunt, which should be fast and concise, and
>>> integrated into explain command. The other is a standalone composer
>>> similar to PigPen, which should be much powerful. PIG-2586 is intended
>>> to track the first, but Russel's comment is talking about the second.
>>> Both are acceptable as a GSoC project. I leave it to Russel.
>>>
>>> >
>>> > (2). Build system
>>> > The code base of my research project is 40K loc and the build script
>>> was
>>> > written in Ant. Part of my duty is to convert the ant build script to
>>> maven
>>> > and maintain the build script. Therefore, Mavenize Pig is of interest
>>> to me
>>> > too. The build.xml in PIG project is more complicated than the one I
>>> worked
>>> > before. It includes ant, maven and ivy. Do we need to use maven to do
>>> all
>>> > the tasks and get rid of all the dependency on ant, maven and ivy?
>>>
>>> Yes
>>>
>>> >
>>> >  Best regards
>>> >  Shasha(Amy) Liu
>>>
>>
>>
>>
>
>
> --
> Russell Jurney twitter.com/rjurney [EMAIL PROTECTED] datasyndrome.
> com
>

--
Shasha(Amy) Liu

Hi Russell and Daniel,

I have set my proposal on google-melange public and available at *
http://www.google-melange.com/gsoc/proposal/review/google/gsoc2012/grassonsnad/4001
*.

I read through the original idea page as well as all the email discussions
concerning with the PIG-2586 project again, and found some problems that
I'm not quite sure about the visualization:

1. input to the visualizer

Is the input a (1).pig script file, or (2) a command entered from a *grunt*
console? If the input is script file which contains a series of operations
for a Pig task, we could visualize the whole plan/data at one time, and
meanwhile provide the ability of step-by-step visualization of individual
operations. If the input is from the *grunt* command line, we need to
update visualization on the fly.

2. content of the visualization

>From the original description of this project and earlier email discussion,
I feel that the visualizer needs to provide information for the 3 layers of
(1)schema, (2)plan and (3)data, collected from DESCRIBE, EXPLAIN and
ILLUSTRATE respectively, all on one canvas, but with the ability of
switching on and off the display of one or two layers. But later
discussions made me feel that we need to separate the visualization of
EXPLAIN and ILLUSTRATE, if so, I am wondering what the updated, detailed
requirements of the visualizer are for the project. (one small question:
does the visualizer need to provide all the three kinds of plan diagrams –
logical, physical and mapreduce plan diagrams, or only the physical plan
diagram as shown in the sigmod09 paper?)

Looking forward to your reply.

Thank you very much.

Regards,

On Fri, Apr 6, 2012 at 1:37 PM, Russell Jurney <[EMAIL PROTECTED]>wrote:

> I can't see the proposal, as I'm not an approved mentor.  I'd suggest
> working with [EMAIL PROTECTED] if possible. You might focus on
> visualization of ILLUSTRATE while he focuses on the editor?
> On Fri, Mar 23, 2012 at 11:21 PM, Shasha Liu <[EMAIL PROTECTED]>wrote:
>
>> Hi Daniel,
>>
>> Thanks a lot for the reply.
>> I installed the latest Pig and read through the book of "programming in
>> pig".
>> I manged to use "-dot -out filename" to produce three graphs in dot file
>> format.
>>
>> Based on the existing dot file, my next question is what is the
>> requirement regarding a better visualizer?
>> Are we going to generate a picture (e.g., .png) for different plans
>> (logical plan, physical plan, map reduce plan), or provide a web interface
>> to visualize these graphs of plans?
>>
>> Best regards,
>>  --
>> Shasha(Amy) Liu
>>
>>
>> On Sun, Mar 18, 2012 at 3:30 AM, Daniel Dai <[EMAIL PROTECTED]>wrote:
>>
>>> See comments inline.
>>>
>>> On Sat, Mar 17, 2012 at 6:52 AM, grassonsand <[EMAIL PROTECTED]>
>>> wrote:
>>> > Dear all,
>>> >
>>> > I am a Ph.D. student in Computer Science and have 4-year Java
>>> programming
>>> > experience focusing on Java Web development.
>>> > In the candidate projects in PIG, I am interested in PIG-2586 (A better
>>> > plan/data flow visualizer) and PIG-2599 (Mavenize Pig).
>>> >
>>> > In my on-going research project, I am in charge of (1). web user
>>> interface
>>> > development and (2). build system. Now I am working on adding hadoop
>>> > capability to the project. The main reason I am interested in the PIG
>>> > project is that I can make a contribution to the PIG community based
>>> on my
>>> > previous experience,  and learn from the participant in GSoC this year
>>> and
>>> > benefit my on-going research project at the same time.
>>> >
>>> > (1). User interface development
>>> > I have used several graphic libraries to visualize semantic data and
>>> our own
>>> > data set, e.g., Jung, graphviz, BIRT, and several plot plugins in
>>> jquery.
>>> > Therefore, I am interested in working on a new tool for PIG visualizer.
>>> > After looking through the bug issue, I have several questions:
>>> >    (i) As both swing and javascript are mentioned, is this project a
>>> web or

Shasha(Amy) Liu