Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - access patterns investigation to dynamically toggle the replication factor in a hadoop cluster


Copy link to this message
-
Re: access patterns investigation to dynamically toggle the replication factor in a hadoop cluster
George Kousiouris 2012-09-05, 16:13

Sorry, something i forgot: are you aware of any available datasets that
could help us in that direction?

On 9/5/2012 7:11 PM, George Kousiouris wrote:
>
> Hi all,
>
> As part of the research for an ongoing project, we are interested in
> investigating the ability  to predict data access patterns on a hadoop
> cluster. The purpose is to study the file access patterns (in a time
> series manner), so that proactive manipulation of data may be
> achieved. This for example may involve the increase/decrease of the
> replication factor in an Apache Hadoop cluster (and according HDFS) to
> deal with an upcoming predicted increase/decrease of data accesses.
>
> So we would like your advise on some issues:
> 1) is this the correct mailing list? :)
> 2) would a changed replication factor translate to a better
> performance of a MR job (either by experience you may have or if you
> have in mind a report/paper etc. that has studied this)
> 3) do you find this interesting in general and something we should
> pursue?
> 4) are you aware of any related work on the topic we could use as a
> starting point?
>
> Thanks for your help,
> George
>
>
>
--

---------------------------

George Kousiouris, PhD
Electrical and Computer Engineer
Division of Communications,
Electronics and Information Engineering
School of Electrical and Computer Engineering
Tel: +30 210 772 2546
Mobile: +30 6939354121
Fax: +30 210 772 2569
Email: [EMAIL PROTECTED]
Site: http://users.ntua.gr/gkousiou/

National Technical University of Athens
9 Heroon Polytechniou str., 157 73 Zografou, Athens, Greece