Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Mumak is still active?


Copy link to this message
-
Re: Mumak is still active?
Nan Zhu 2010-10-29, 02:04
Hi,

Hong, Thank you for the reply,

Actually I'm doing my research on Mumak before I move to a real cluster,
since we are just some guys in university, so we cannot afford a large
cluster containing more than 1000 nodes, I modified and enhanced the code of
Mumak,

And thank you for the reminding me the efficient ways in working at a open
source project, I'll create and release my solutions soon,

Here is a question about the mumak, I think it's really strange:

1. in the class of SImulatroEventQueue, the funtion of add() and addall()
both limit something like this :

  if (e.getTimeStamp() < lastTimeStamp) {
        throw new IllegalArgumentException("Event happens in the past: "
            + e.getClass() + "(" + e.getTimeStamp() + "<" + lastTimeStamp
            + "    Last Event:    " + lastEvent.getClass());

But I find something very strange,

I added a line of code in the createTaskCompletionAttemption in the
SimulatorTaskTracker class:

 LOG.info("TrackerName: " + this.taskTrackerName + "taskid:" +
            status.getTaskID() + "start_time:" + status.getStartTime() +
            "finish_time: " + status.getFinishTime());

so the program will output the information of each taskCompletionEvent,

I got something like :

---------------------------------
10/10/29 09:48:26 INFO mapred.SimulatorTaskTracker: TrackerName:
tracker_node055:localhost/127.0.0.1:10004taskid:attempt_201010132054_0019_r_000000_0start_time:1288317615942finish_time:
1288317789309
10/10/29 09:48:26 INFO mapred.SimulatorTaskTracker: TrackerName:
tracker_node055:localhost/127.0.0.1:10004taskid:attempt_201010132054_0019_r_000001_0start_time:1288317618947finish_time:
1288317720289
---------------------------------
the finish-time of attempt_201010132054_0019_r_000000_0 is 1288317789309,
and attempt_201010132054_0019_r_000001_0 is 1288317720289

the order shows that the first event about
attempt_201010132054_0019_r_000000_0 is in front of the later one in the
event queue, but the later one finishes **earlier** that the prior one, and
the timestamp of completionevent is the finishtime of each event, why it
hasn't throw a IllegalArgumentException I showed above?

anyone can give some explanation?

Thank you

Nan
On Fri, Oct 29, 2010 at 9:28 AM, Hong Tang <[EMAIL PROTECTED]> wrote:

> Nan,
>
> We (at Yahoo) are still doing some work in and on top of Mumak in Yahoo,
> but not in significant ways because most of our production is using an
> internal version of Hadoop 20, which diverges from Hadoop trunk (where Mumak
> resides). So the project is certainly "live" but not very "active".
>
> I guess you could look at it this way: would Mumak be a good base to help
> you accelerate what you plan to work on? If the answer is yes, take it and
> enhance it. Just like any open source projects, there is no particular owner
> of it. The more people embrace it and contribute to it, the more likely the
> project will be successful.
>
> Another note I should say is that before you invest in significant amount
> of time in implementing something, you should create a jira ticket first,
> and discuss solutions with the community (sometimes even post partial
> patches) along the way. Sometimes your solution may not be aligned with the
> general design principle of the whole project, or some assumptions you make
> may no longer hold in future versions. Keep the community involved helps
> avoid such costly mistakes.
>
> Hope this helps,
>
> -Hong
>
>
> On Oct 27, 2010, at 11:52 PM, Nan Zhu wrote:
>
>  Hi, all
>>
>> I'm working with Mumak recently, but I found that the code hasn't been
>> updated for a long while, I would like to know that if the Mumak is still
>> active?
>>
>>
>> Thank you
>>
>> Nan
>>
>
>