Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Blur >> mail # dev >> Optimizing for indexing vs searching


+
rahul challapalli 2014-04-08, 00:36
+
Aaron McCurry 2014-04-08, 02:10
+
rahul challapalli 2014-04-08, 14:55
Copy link to this message
-
Re: Optimizing for indexing vs searching
On Tue, Apr 8, 2014 at 10:54 AM, rahul challapalli <
[EMAIL PROTECTED]> wrote:

You are welcome!  :-)

Ok, with that kind of hardware (assuming you have enough nodes) you should
have plenty of headroom for indexing and search.  With small document
(record) sizes around 1K worth of information a single cluster of 32 nodes
can easily keep up with 10 to 20K records per second for hours at a time
(with a single client writing the updates).  The indexing speed greatly
depends on the information you are indexing, so this is something you will
likely have to try out to really see what it's capable of achieving.  Let
me know if I can help in anyway.

That's the one.  You can configure the number of work threads per shard
process.

Aaron