Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> Cyclic dependency in JobControl job DAG


Copy link to this message
-
Re: Cyclic dependency in JobControl job DAG

+1, can you please open a JIRA ticket (at https://issues.apache.org/jira/browse/MAPREDUCE) and attach a patch?

Thanks,
+Vinod

On Jun 25, 2012, at 6:53 AM, Robert Evans wrote:

> I personally think it is useful.  I would say contribute it.
>
> (Moved common-dev to bcc, we try not to cross post on these lists)
>
> --Bobby Evans
>
> On 6/25/12 3:37 AM, "madhu phatak" <[EMAIL PROTECTED]> wrote:
>
> Hi,
> In current implementation of JobControl, whenever there is a cyclic
> dependency between the jobs it throws a Stack overflow exception .
> For example,
>   ControlledJob job1 = new ControlledJob(new Configuration());
> job1.setJobName("job1");
> ControlledJob job2 = new ControlledJob(new Configuration());
> job2.setJobName("job2");
> job1.addDependingJob(job2);
> job2.addDependingJob(job1);
>                JobControl jobControl = new JobControl("jobcontrol");
> jobControl.addJob(job1);
> jobControl.addJob(job2);
> jobControl.run();
>
> throws
>  java.lang.StackOverflowError
> at java.util.ArrayList.get(ArrayList.java:322)
> at
> org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.checkState(ControlledJob.java:295)
>
> Whenever we write complex application, there is always possibility of
> cyclic dependencies.I have written a method which checks for the cyclic
> dependency  upfront and informs it to the user. I want to know from you
> guys, do you think is it a useful feature? If yes I can contribute it as a
> patch.
>
> Regards,
> Madhukara Phatak
> --
> https://github.com/zinnia-phatak-dev/Nectar
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB