Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> for UDF, figure out whether it's on a task tracker?


Copy link to this message
-
for UDF, figure out whether it's on a task tracker?
normally job tracker and task tracker is on different nodes.

when I submit a pig script using UDF. I think the UDF constructor is first
run (several times, don't know why)
on the job tracker, and then it's run on each of the task trackers.

now I want to do some custom work inside the constructor, such as checking
the existence of certain files
which are specific to only task trackers. such work only needs to be done
on task trackers.
So , is there a way to figure out whether the UDF is being run on task
tracker or job tracker?

Thanks!
yang
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB