Hello everyone,

I am looking into curator service discovery API to implement a service that
meets some requirements. I know there are three strategies to retrieve a
service instance: Round-robin, Random, and Sticky. I want something
different than these strategies.

Suppose I have 3 service instance with id of 3, 5, 8. Client does not have
any information about these ids. When client request an instance with id of
5 "getNewInstance(5)", the service should return the instance whose id is 5.
This is OK since we have it in our repository. However, when client issues
"getNewInstance(6)", the service provider should return the instance whose
id is 8, not an error. I mean the service provider should return the
instance whose is the next smallest one in the repository. I do not know if
this is possible with the current strategies.

Any help/comment is appreciated.

best
Yasin

--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

FYI - Curator has its own mailing list/group: http://groups.google.com/group/curator-users

What you want should be easy to do. Curator-Discovery supports custom ProviderStrategy-s. Just write one that does what you describe. The instance id is available from ServiceInstance.getId() or you can store an index in the ServiceInstance's payload.

-Jordan

On Jan 28, 2013, at 12:10 PM, Yasin <[EMAIL PROTECTED]> wrote:

> Hello everyone,
>
> I am looking into curator service discovery API to implement a service that
> meets some requirements. I know there are three strategies to retrieve a
> service instance: Round-robin, Random, and Sticky. I want something
> different than these strategies.
>
> Suppose I have 3 service instance with id of 3, 5, 8. Client does not have
> any information about these ids. When client request an instance with id of
> 5 "getNewInstance(5)", the service should return the instance whose id is 5.
> This is OK since we have it in our repository. However, when client issues
> "getNewInstance(6)", the service provider should return the instance whose
> id is 8, not an error. I mean the service provider should return the
> instance whose is the next smallest one in the repository. I do not know if
> this is possible with the current strategies.
>
> Any help/comment is appreciated.
>
> best
> Yasin
>
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.

Thank you for the quick response. I joined the curator mailing list.

For every request, should I retrieve all the ids? Do you think this will be
overhead in case I have 10K or more instances?

Thanks

--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578450.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

The IDs are retrieved for you.

10K instances of a single type? You might run into problems with that. Can you sub-divide them in any way? ZooKeeper doesn't handle ZNodes with that many children very well.

-JZ

On Jan 28, 2013, at 12:41 PM, Yasin <[EMAIL PROTECTED]> wrote:

> Thank you for the quick response. I joined the curator mailing list.
>
> For every request, should I retrieve all the ids? Do you think this will be
> overhead in case I have 10K or more instances?
>
> Thanks
>
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578450.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.

We will build this service for location discovery. The service we want to
build will act like a DNS server. The service will provide a node IP based
on some given information. I assume that a data center may have 10K or even
more nodes in it. So any client who wants to store or retrieve a file should
find a node in the data center. At most, we can classify nodes into clusters
or racks.  

Suppose, we have 500 racks and each has 32 nodes, that makes 16K nodes.
Zookeeper will store 500 Znoodes and each znode will store 32 znodes. For
example the znodes structure will look like root/rack1/node1,
root/rack1/node2..., root/rack2/node1.., and so on. I do not know if this a
good approach for such a service. Or do you think I shouldn't use zookeeper
and curator for this service at all and look for some other solutions?

Thanks
Yasin  

--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578452.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

So, each rack is a different "service" in the Curator-Discovery sense? If so, that will work very well. You should avoid situations where a getChildren() call will approach 1MB of data.

-JZ

On Jan 28, 2013, at 1:15 PM, Yasin <[EMAIL PROTECTED]> wrote:

> We will build this service for location discovery. The service we want to
> build will act like a DNS server. The service will provide a node IP based
> on some given information. I assume that a data center may have 10K or even
> more nodes in it. So any client who wants to store or retrieve a file should
> find a node in the data center. At most, we can classify nodes into clusters
> or racks.  
>
> Suppose, we have 500 racks and each has 32 nodes, that makes 16K nodes.
> Zookeeper will store 500 Znoodes and each znode will store 32 znodes. For
> example the znodes structure will look like root/rack1/node1,
> root/rack1/node2..., root/rack2/node1.., and so on. I do not know if this a
> good approach for such a service. Or do you think I shouldn't use zookeeper
> and curator for this service at all and look for some other solutions?
>
> Thanks
> Yasin  
>
>
>
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578452.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.

This will work if we think each rack as a different service. But now the
service retrieval becomes problematic. Event though I think each service as
a different service, I should think all the service as a single service. We
still need to keep on that id numbering thing. I mean rack1 might have ids
from 1..32, and rack2 might have 33..64. So I still need to find a node
whose id is best for the client, either has the same number or next smallest
one.
Another approach would be to classify nodes into, say 10 clusters, and put
each node in an appropriate znode. For example,
root/cluster1_1000/node1,...,root/cluster1_1000/node1000,  
root/cluster1001_2000/node1,...,root/cluster1001_2000/node1000, and soon. If
the service gets "getInstance(245)" it will know that the desired node will
be in root/cluster1001_2000/. Then I will use some heuristics to retrieve
some nodes, sort them based on their ids, and find the most appropriate one.
How about this idea?

Thanks

Yasin

--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578456.html
Sent from the zookeeper-user mailing list archive at Nabble.com.

It's really up to you. The only thing to be concerned with is the Jute transport limits. Of course, you can always increase this. See jute.maxbuffer here: http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html

For Curator-Discovery you can calculate the space you need. See here: https://github.com/Netflix/curator/wiki/Service-Discovery - Curator-Discovery writes nodes with their UUID name. Each UUID node will be 36 bytes. So, a getChildren() call on 10K nodes is 10K as 36 or ~360K. So, you're well under the 1MB limit there. The main issue is if you use a ServiceCache the initial load will require 10K+ ZK calls. This will probably be acceptable. On a gigabit network this won't take too long. Bear in mind that there will also be 10K+ watchers in ZK. Again, ZK should handle this fine.

Do you expect to grow an order of magnitude from 10K? If so, you might consider clustering. If 10K is the limit you should be fine.

-JZ

On Jan 28, 2013, at 3:34 PM, Yasin <[EMAIL PROTECTED]> wrote:

> This will work if we think each rack as a different service. But now the
> service retrieval becomes problematic. Event though I think each service as
> a different service, I should think all the service as a single service. We
> still need to keep on that id numbering thing. I mean rack1 might have ids
> from 1..32, and rack2 might have 33..64. So I still need to find a node
> whose id is best for the client, either has the same number or next smallest
> one.
> Another approach would be to classify nodes into, say 10 clusters, and put
> each node in an appropriate znode. For example,
> root/cluster1_1000/node1,...,root/cluster1_1000/node1000,  
> root/cluster1001_2000/node1,...,root/cluster1001_2000/node1000, and soon. If
> the service gets "getInstance(245)" it will know that the desired node will
> be in root/cluster1001_2000/. Then I will use some heuristics to retrieve
> some nodes, sort them based on their ids, and find the most appropriate one.
> How about this idea?
>
> Thanks
>
> Yasin
>
>
>
> --
> View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578456.html
> Sent from the zookeeper-user mailing list archive at Nabble.com.

Thinking more about your usage…

It sounds like you will use your own ID scheme so the payload for getChildren() will be much smaller. This is probably more do-able than I originally thought. Further, if you run into performance problems you can decide to not use a ServiceProvider (which uses a cache internally). Instead, you can query directly each time by calling:

ServiceDiscovery.queryForInstance(String name, String id)

You can treat this as a binary search. If the id you try isn't available, search again using a bsearch algorithm.

Just a thought…

-JZ

On Jan 28, 2013, at 3:43 PM, Jordan Zimmerman <[EMAIL PROTECTED]> wrote:

> It's really up to you. The only thing to be concerned with is the Jute transport limits. Of course, you can always increase this. See jute.maxbuffer here: http://zookeeper.apache.org/doc/r3.4.5/zookeeperAdmin.html
>
> For Curator-Discovery you can calculate the space you need. See here: https://github.com/Netflix/curator/wiki/Service-Discovery - Curator-Discovery writes nodes with their UUID name. Each UUID node will be 36 bytes. So, a getChildren() call on 10K nodes is 10K as 36 or ~360K. So, you're well under the 1MB limit there. The main issue is if you use a ServiceCache the initial load will require 10K+ ZK calls. This will probably be acceptable. On a gigabit network this won't take too long. Bear in mind that there will also be 10K+ watchers in ZK. Again, ZK should handle this fine.
>
> Do you expect to grow an order of magnitude from 10K? If so, you might consider clustering. If 10K is the limit you should be fine.
>
> -JZ
>
> On Jan 28, 2013, at 3:34 PM, Yasin <[EMAIL PROTECTED]> wrote:
>
>> This will work if we think each rack as a different service. But now the
>> service retrieval becomes problematic. Event though I think each service as
>> a different service, I should think all the service as a single service. We
>> still need to keep on that id numbering thing. I mean rack1 might have ids
>> from 1..32, and rack2 might have 33..64. So I still need to find a node
>> whose id is best for the client, either has the same number or next smallest
>> one.
>> Another approach would be to classify nodes into, say 10 clusters, and put
>> each node in an appropriate znode. For example,
>> root/cluster1_1000/node1,...,root/cluster1_1000/node1000,  
>> root/cluster1001_2000/node1,...,root/cluster1001_2000/node1000, and soon. If
>> the service gets "getInstance(245)" it will know that the desired node will
>> be in root/cluster1001_2000/. Then I will use some heuristics to retrieve
>> some nodes, sort them based on their ids, and find the most appropriate one.
>> How about this idea?
>>
>> Thanks
>>
>> Yasin
>>
>>
>>
>> --
>> View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578456.html
>> Sent from the zookeeper-user mailing list archive at Nabble.com.
>

We are planning to go beyond 10K nodes even geographically distributed data
centers.
We will use our own ID scheme and it will be something between 1-2^32
(Integer) or maybe later long number. So doing binary search might not be
feasible in terms of performance. I would do binary search on a part of
nodes, say 1000 out of 10K or more.

We are at the design stage, therefore we want to find the best way to
implement this service so that it will meet our today's and possible future
needs.

Thanks

Yasin

--
View this message in context: http://zookeeper-user.578899.n2.nabble.com/curator-service-discovery-search-to-Select-a-service-intance-tp7578447p7578461.html
Sent from the zookeeper-user mailing list archive at Nabble.com.