Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> 0 tasktrackers in jobtracker but all datanodes present


Copy link to this message
-
Re: 0 tasktrackers in jobtracker but all datanodes present
Gaurav
       NN memory might have hit its upper bound. As a bench mark, for every
1 million files/blocks/directories 1GB of memory is required on the NN. The
number of files in your cluster might have grown beyond this treshold. So
the options left for you would be
- If there are large number of small files, use HAR or Sequence File for
grouping the same
- Increase the NN heap

Regards
Bejoy KS

On Mon, Apr 2, 2012 at 12:08 PM, madhu phatak <[EMAIL PROTECTED]> wrote:

> Hi,
> 1. Stop the job tracker and task trackers.  - bin/stop-mapred.sh
>
>  2. Disable namenode safemode - bin/hadoop dfsadmin -safemode leave
>
> 3. Start the job tracker and tasktrackers again - bin/start-mapred.sh
>
> On Fri, Jan 13, 2012 at 5:20 AM, Ravi Prakash <[EMAIL PROTECTED]>
> wrote:
>
> > Courtesy Kihwal and Bobby
> >
> > Have you tried increasing the max heap size with -Xmx? and make sure that
> > you have swap enabled.
> >
> > On Wed, Jan 11, 2012 at 6:59 PM, Gaurav Bagga <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi
> > >
> > > hadoop-0.19
> > > I have a working hadoop cluster which has been running perfectly for
> > > months.
> > > But today after restarting the cluster, at jobtracker UI its showing
> > state
> > > INITIALIZING for a long time and is staying on the same state.
> > > The nodes in jobtracker are zero whereas all the nodes are present on
> the
> > > dfs.
> > > It says Safe mode is on.
> > > grep'ed on slaves and I see the tasktrackers running.
> > >
> > > In namenode logs i get the following error
> > >
> > >
> > > 2012-01-11 16:50:57,195 WARN  ipc.Server - Out of Memory in server
> select
> > > java.lang.OutOfMemoryError: Java heap space
> > >        at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:39)
> > >        at java.nio.ByteBuffer.allocate(ByteBuffer.java:312)
> > >        at
> > > org.apache.hadoop.ipc.Server$Connection.readAndProcess(Server.java:804)
> > >        at org.apache.hadoop.ipc.Server$Listener.doRead(Server.java:400)
> > >        at org.apache.hadoop.ipc.Server$Listener.run(Server.java:309)
> > >
> > > Not sure why the cluster is not coming up
> > > -G
> > >
> >
>
>
>
> --
> https://github.com/zinnia-phatak-dev/Nectar
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB