Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> secondary namenode is hang at post


Copy link to this message
-
Re: secondary namenode is hang at post
It looks that it can not copy the new checkpoint into the NameNode. Can you
copy-past what jstack says?

$ sudo -u hdfs jstack <snn-pid>

2013/12/11 Patai Sangbutsarakum <[EMAIL PROTECTED]>

> It just happens without changing anything in the cluster. Secondary
> namenode node has been working fine until today i notice that in second
> namenode log file stop at.
>
> 2013-12-11 13:17:41,083 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Image file of size 3941631662 saved in 61 seconds.
> 2013-12-11 13:19:15,446 INFO org.apache.hadoop.hdfs.server.common.Storage:
> Image file of size 3941631662 saved in 94 seconds.
> 2013-12-11 13:19:29,760 INFO
> org.apache.hadoop.hdfs.server.namenode.SecondaryNameNode: Posted URL
> primarynn:50070putimage=1&port=50090&machine=snn2&token=-19:1944437056:1306462484827:1386562944000:1386559939508&newChecksum=2ad3e008becbd2cd5196b2b6ba1498ac
>
>
> #date
> Wed Dec 11 14:16:30 PST 2013
>
> even after 1 hour passed it's not finish doing the checkpoint. looking at
> timestamp of current/fsimage.ckpt at primary namenode; it doesn't show
> progress in size and timestamp of the file is days ago.
>
> already tried to clean the current in snn and restart secondarynamenode
> process, but SNN still stop at the same spot even thought the snn process
> is still exist.
>
>
> -P
>