Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns


+
kayhan 2012-04-10, 13:45
Copy link to this message
-
Re: FPGrowth job got stuck when running fpGrowth.generateTopKFrequentPatterns
Jay Vyas 2012-04-10, 15:28
Do you have counters for the reducers , and if so , are they continuing to
update ?

On Tue, Apr 10, 2012 at 8:45 AM, kayhan <[EMAIL PROTECTED]> wrote:

> Trying to mine frequent patterns of a dataset with 8000 transactions and
> 193
> attributes using mahout's parallel-frequent-pattern-mining algorithm. I run
> it in map-reduce mode on a cluster of 10 machines on windows7 and I use
> cygwin.
>
> When I run it, in reduce step, it shows completion of 100%, however it
> stucks there. No error or any other indication, but it waits there for
> hours
> and does not finish working.
> The details of the job shows that, it can not finish the part 'Processing
> FPTree: Bottom Up FP Growth > reduce'. For example the part 'Writing Top K
> patterns for: 338 > reduce' is finished in seconds.
> I post sample image of the gui that shows running tasks;
>
> http://hadoop-common.472056.n3.nabble.com/file/n3899658/Ads%C4%B1z2.png
>
> Also a part of the tasktracker log of the slave that is responsible for the
> job is posted below;
>
> /
> 2012-04-10 16:07:11,051 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:20,084 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:32,127 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:41,160 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:07:50,192 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:02,235 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> 2012-04-10 16:08:11,268 INFO org.apache.hadoop.mapred.TaskTracker:
> attempt_201204101532_0003_r_000014_1 1.0% Processing FPTree: Bottom Up FP
> Growth > reduce
> /
>
> If have any suggestions would be glad.
>
>
> --
> View this message in context:
> http://hadoop-common.472056.n3.nabble.com/FPGrowth-job-got-stuck-when-running-fpGrowth-generateTopKFrequentPatterns-tp3899658p3899658.html
> Sent from the Users mailing list archive at Nabble.com.
>

--
Jay Vyas
MMSB/UCHC
+
kayhan 2012-04-10, 16:00