Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> java.net.SocketException: Too many open files


Copy link to this message
-
java.net.SocketException: Too many open files

hi, all:
        I had met a question after my program continued for 28+ hours under the circumstances of cluster which have three machine that had set ulimit to 32K.............2010-06-13 01:42:57,318 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftrail=5003\x253Acat10361\x26catId=cat10361\x26type=category\x26addFacet=5001\x253Ab211,1276366391680 because global memstore limit of 395.6m exceeded; currently 372.2m and flushing till 346.1m2010-06-13 01:42:57,630 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.mwave.www:http/mwave/subcategory.asp\x3FCatID=169\x26parent=3\x5E165,1276303221573 because global memstore limit of 395.6m exceeded; currently 349.0m and flushing till 346.1m2010-06-13 01:42:57,949 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtabletest,com.teamstore.www:http/product/index.jsp\x3F
 productId=3425980\x26cp=1209615.714811\x26parentPage=family,12763421607472010-06-13 01:42:57,961 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.teamstore.www:http/product/index.jsp\x3FproductId=3425980\x26cp=1209615.714811\x26parentPage=family,1276342160747 in 0sec2010-06-13 01:42:57,962 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftrail=5003\x253Acat10361\x26catId=cat10361\x26type=category\x26addFacet=5001\x253Ab211,12763663916802010-06-13 01:42:57,973 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftrail=5003\x253Acat10361\x26catId=cat10361\x26type=category\x26addFacet=5001\x253Ab211,1276366391680 in 0sec2010-06-13 01:42:57,973 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtable
 test,com.mwave.www:http/mwave/subcategory.asp\x3FCatID=169\x26parent=3\x5E165,12763032215732010-06-13 01:43:03,360 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@1fe8d892010-06-13 01:43:05,522 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@109fc12010-06-13 01:43:05,583 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@57ebd62010-06-13 01:43:07,716 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@8e8ab52010-06-13 01:43:07,751 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@146d4d92010-06-13 01:43:07,966 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@b1258d2010-06-13 01:43:08,061 WARN org.apache.hadoop.hbase.regionser
 ver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@8fa7d42010-06-13 01:43:08,130 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@c232962010-06-13 01:43:08,171 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@84add62010-06-13 01:43:08,269 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@8a7a3a2010-06-13 01:43:08,337 WARN org.apache.hadoop.hbase.regionserver.Store: Not in setorg.apache.hadoop.hbase.regionserver.StoreScanner@cd86a52010-06-13 01:43:08,345 INFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.mwave.www:http/mwave/subcategory.asp\x3FCatID=169\x26parent=3\x5E165,1276303221573 in 10sec2010-06-13 02:06:12,436 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.cableorganizer:http
 /briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,1276264102072 because global memstore limit of 395.6m exceeded; currently 395.6m and flushing till 346.1m2010-06-13 02:06:12,811 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.pacsun.shop:http/js_external/sj_flyout.js,1276322851971 because global memstore limit of 395.6m exceeded; currently 372.5m and flushing till 346.1m2010-06-13 02:06:13,136 INFO org.apache.hadoop.hbase.regionserver.MemStoreFlusher: Forced flushing of nutchtabletest,com.vitaminshoppe.www:http/search/en/category.jsp\x3Ftype=category\x26catId=cat10134,1276366391680 because global memstore limit of 395.6m exceeded; currently 350.3m and flushing till 346.1m2010-06-13 02:06:13,562 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting compaction on region nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,12762641020722010-06-13 02:06:14,175 I
 NFO org.apache.hadoop.hbase.regionserver.HRegion: compaction completed on region nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,1276264102072 in 0sec2010-06-13 02:06:14,175 INFO org.apache.hadoop.hbase.regionserver.HRegion: Starting split of region nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,12762641020722010-06-13 02:06:14,178 INFO org.apache.hadoop.hbase.regionserver.HRegion: Closed nutchtabletest,com.cableorganizer:http/briggs-stratton-generators/storm-ready-kit.htm\x3F=recommended,12762641020722010-06-13 02:06:14,710 INFO org.apache.hadoop.hbase.regionserver.CompactSpl