Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> java.net.SocketException: Too many open files


Copy link to this message
-
java.net.SocketException: Too many open files

hi, all:
        I had met a question after my program continued for 28+ hours under the circumstances of cluster which have three machine that had set ulimit to 32K.............2010-06-13 01:42:57,318 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftrail=5003\x253Acat10361\x26catId=cat10361\x26type=category\x26addFacet=5001\x253Ab211,1276366391680 because global memstore limit of 395.6m exceeded; currently 372.2m and flushing till 346.1m2010-06-13 01:42:57,630 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.mwave.www:http/mwave/subcategory.asp\x3FCatID=169\x26parent=3\x5E165,1276303221573 because global memstore limit of 395.6m exceeded; currently 349.0m and flushing till 346.1m2010-06-13 01:42:57,949 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtabletest,com.teamstore.www:http/product/index.jsp\x3F
 productId=3425980\x26cp=1209615.714811\x26parentPage=family,12763421607472010-06-13 01:42:57,961 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.teamstore.www:http/product/index.jsp\x3FproductId=3425980\x26cp=1209615.714811\x26parentPage=family,1276342160747 in 0sec2010-06-13 01:42:57,962 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftrail=5003\x253Acat10361\x26catId=cat10361\x26type=category\x26addFacet=5001\x253Ab211,12763663916802010-06-13 01:42:57,973 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftrail=5003\x253Acat10361\x26catId=cat10361\x26type=category\x26addFacet=5001\x253Ab211,1276366391680 in 0sec2010-06-13 01:42:57,973 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtable
 test,com.mwave.www:http/mwave/subcategory.asp\x3FCatID=169\x26parent=3\x5E165,12763032215732010-06-13 01:43:03,360 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@1fe8d892010-06-13 01:43:05,522 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@109fc12010-06-13 01:43:05,583 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@57ebd62010-06-13 01:43:07,716 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@8e8ab52010-06-13 01:43:07,751 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@146d4d92010-06-13 01:43:07,966 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@b1258d2010-06-13 01:43:08,061 WARN org.apache.hadoop.hbase.regionser
 ver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@8fa7d42010-06-13 01:43:08,130 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@c232962010-06-13 01:43:08,171 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@84add62010-06-13 01:43:08,269 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@8a7a3a2010-06-13 01:43:08,337 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@cd86a52010-06-13 01:43:08,345 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.mwave.www:http/mwave/subcategory.asp\x3FCatID=169\x26parent=3\x5E165,1276303221573 in 10sec2010-06-13 02:06:12,436 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.cableorganizer:http
 /briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,1276264102072 because global memstore limit of 395.6m exceeded; currently 395.6m and flushing till 346.1m2010-06-13 02:06:12,811 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.pacsun.shop:http/js_external/sj_flyout.js,1276322851971 because global memstore limit of 395.6m exceeded; currently 372.5m and flushing till 346.1m2010-06-13 02:06:13,136 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftype=category\x26catId=cat10134,1276366391680 because global memstore limit of 395.6m exceeded; currently 350.3m and flushing till 346.1m2010-06-13 02:06:13,562 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,12762641020722010-06-13 02:06:14,175 I
 NFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,1276264102072 in 0sec2010-06-13 02:06:14,175 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting split of region nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,12762641020722010-06-13 02:06:14,178 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,12762641020722010-06-13 02:06:14,710 INFO org.apache.hadoop.hbase.regionserver.CompactSpl
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB