Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns


Copy link to this message
-
Re: FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns
Do you have counters for the reducers , and if so , are they continuing to
update ?

On Tue, Apr 10, 2012 at 8:45 AM, kayhan <[EMAIL PROTECTED]> wrote:

> Trying to mine frequent patterns of a dataset with 8000 transactions and
> 193
> attributes using mahout's parallel-frequent-pattern-mining algorithm. I run
> it in map-reduce mode on a cluster of 10 machines on windows7 and I use
> cygwin.
>
> When I run it, in reduce step, it shows completion of 100%, however it
> stucks there. No error or any other indication, but it waits there for
> hours
> and does not finish working.
> The details of the job shows that, it can not finish the part 'Processing
> FPTree: Bottom Up FP Growth > reduce'. For example the part 'Writing Top K
> patterns for: 338 > reduce' is finished in seconds.
> I post sample image of the gui that shows running tasks;
>
> http://hadoop-common.472056.n3.nabble.com/file/n3899658/Ads%C4%B1z2.png
>
> Also a part of the tasktracker log of the slave that is responsible for the
> job is posted below;
>
> /
> 2012-04-10 16:07:11,051 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:20,084 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:32,127 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:41,160 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:50,192 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:02,235 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:11,268 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> /
>
> If have any suggestions would be glad.
>
>
> --
> View this message in context:
> http://hadoop-common.472056.n3.nabble.com/FPGrowth-job-got-stuck-when-running-fpGrowth-generateTopKFrequentPatterns-tp3899658p3899658.html
> Sent from the Users mailing list archive at Nabble.com.
>

--
Jay Vyas
MMSB/UCHC
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB