Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - debug pig UDF in production EC2 using eclipse ?

Copy link to this message
debug pig UDF in production EC2 using eclipse ?
Yang 2012-02-03, 06:01
sometimes I need to debug a pig UDF in production EC2 cluster using eclipse,

this is normally what I do for debugging a remote jvm:

add "-Xdebug -Xrunjdwp:transport=dt_socket,address=12345,server=y,suspend=y
" to JVM options

then connect eclipse "debug remote application" to this remote host and
port 12345

but EC2 normally does not open any ports, so I do ssh tunneling from my
laptop (where eclipse runs) to EC2,
and then tell eclipse to connect to localhost:12345  ( the ssh command is
"ssh -L12345:ec2_box:12345 ec2_box )

the above procedure works perfect for normal java jobs. but somehow it
takes a long while for pig (running in -x local mode ) to respond ---
although it does finally connect to eclipse
how do you handle such cases?