Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> How to run Fault injection in HDFS


Copy link to this message
-
Re: How to run Fault injection in HDFS
Generally the idea was to provide everything needed for injection by what
current build.xml is having in Common and Hdfs. Would you mind to share what
extra changes you've needed and why?

Cos

On 11/20/09 12:32 , Thanh Do wrote:
> Thank you folks!
>
> Finally, I am able (really) to run FI with HADOOP. I added some aspects
> into the source code, changed the build.xml, and that's it.
>
> AspectJ is awesome!
>
> Have a nice weekend!
>
> On Fri, Nov 20, 2009 at 1:08 PM, Konstantin Boudnik <[EMAIL PROTECTED]
> <mailto:[EMAIL PROTECTED]>> wrote:
>
>     Hi Thanh.
>
>     hmm, it sounds like you have some issue with compilation of your code.
>
>     addDeprication() has been added to Configuration in 0.21, I believe.
>     And it is there no matter how do you compile your code (with FI or
>     without).
>
>     Cos
>
>
>     On 11/19/09 10:12 , Thanh Do wrote:
>
>         Sorry to dig this thread again!
>
>         I am expecting the release of 0.21 so that I don't have to
>         manually play
>         around with AspectJ FI any more.
>
>         I still have problem with running HDFS with instrumented code
>         (with aspect).
>
>         Here is what I did:
>
>         In the root directory of HDFS:
>         /$ ant injectfaults
>
>         $ ant jar-fault-inject
>         /At this point, i have a jar file containing hdfs classed, namely,
>         /hadoop-hdfs-0.22.0-dev-fi.jar/, located in /build-fi/ folder.
>
>         Now I go to the HADOOP folder (which contains running script in bin
>         directory), and do the following
>         /$ ant compile-core-classes/
>         ( now I need additional hdfs classes to be able to run
>         /start-dfs.sh/ <http://start-dfs.sh/>,
>         right)
>         What I did is copying
>         /$HDFS/build-fi/hadoop-hdfs-0.22.0-dev-fi.jar /to
>         /$HADOOP/hadoop-hdfs-fi-core.jar/ (I need to add suffix "core"
>         since the
>         script will include all hadoop-*-core.jar in classpath)
>
>         /$ bin/start-dfs.sh/ <http://start-dfs.sh/>
>         and got error message:
>
>         2009-11-19 11:52:57,479 ERROR
>         org.apache.hadoop.hdfs.server.namenode.NameNode:
>         java.lang.NoSuchMethodError:
>         org.apache.hadoop.conf.Configuration.addDeprecation(Ljava/lang/String;[Ljava/lang/String;)V
>                  at
>         org.apache.hadoop.hdfs.HdfsConfiguration.deprecate(HdfsConfiguration.java:44)
>                  at
>         org.apache.hadoop.hdfs.HdfsConfiguration.addDeprecatedKeys(HdfsConfiguration.java:48)
>                  at
>         org.apache.hadoop.hdfs.HdfsConfiguration.<clinit>(HdfsConfiguration.java:28)
>                  at
>         org.apache.hadoop.hdfs.server.namenode.NameNode.createNameNode(NameNode.java:1169)
>                  at
>         org.apache.hadoop.hdfs.server.namenode.NameNode.main(NameNode.java:1199)
>
>         2009-11-19 11:52:57,480 INFO
>         org.apache.hadoop.hdfs.server.namenode.NameNode: SHUTDOWN_MSG:
>
>         Could any one tell me how to solve this problem?
>
>         Thank you so much.
>
>
>         On Thu, Oct 8, 2009 at 10:41 AM, Konstantin Boudnik
>         <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>
>         <mailto:[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>> wrote:
>
>             Thanks for looking into fault injection - it's very
>         interesting and
>             useful technique based on AspectJ.
>
>             Currently, it is fully integrated into HDFS only. There's a JIRA
>             (HADOOP-6204) which tracks the same effort for Common and
>         then all
>             Hadoop's components will have injection (as well as fault
>         injection)
>             in place. This JIRA should be committed in the matter of a
>         couple of
>             weeks.
>
>             For the immediate purpose you don't need to patch anything
>         or do any
>             tweaking of the code: the fault injection framework is in
>         already
>             and ready to work.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB