Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Drill >> mail # user >> Distributed mode troubles: ZK/Curator connection time out


Copy link to this message
-
Re: Distributed mode troubles: ZK/Curator connection time out
Perfect, thank you very much Steven! Will give it a try first thing tomorrow morning.

Cheers,
             Michael

Sent from my iPad

--
Michael Hausenblas, http://mhausenblas.info

> On 27 Oct 2013, at 22:35, Steven Phillips <[EMAIL PROTECTED]> wrote:
>
> As for the submit_plan tool, unfortunately at the time of the M1 release,
> the tool was not fully baked. So it does not have very good parsing
> capabilities. But as you wrote in your diagram, it takes three parameters,
> a file, a type, and a zk connect string. The file is a path on the local
> filesystem to a file that contains either a logical or a physical plan, in
> json format. The type should be either physical or logical, and this
> corresponds to the type of plan that the file parameter points to. And the
> zk connect string is the zk connect string for the cluster you want to
> submit to.
>
> for example:
>
> submit_plan /plans/plan1.json physical 1.1.1.1:5181,2.2.2.2:5181 will use
> the zk quorum of servers running on 1.1.1.1 and 2.2.2.2 to find and connect
> to a drillbit, and then submit the physical plan contained in
> /plans/plan1.json to the drillbit. It will then print results to the screen
> as they come in.
>
> If you build drill off of master, the submit_plan tool has been fixed
> somewhat:
>
> $ bin/submit_plan -h
>
> Usage: ./submit_plan [options]
>  Options:
>    -h, -help, --help
>       show usage
>       Default: false
>    -bits
>       number of drillbits to run. local mode only
>       Default: 1
>  * -f
>       file containing plan
>    -local
>       run query in local mode
>       Default: false
>  * -t
>       type of plan, logical/physical
>    -zk
>       zookeeper connect string.
>       Default: localhost:2181
>
>
>
> On Sun, Oct 27, 2013 at 3:17 PM, Steven Phillips <[EMAIL PROTECTED]>wrote:
>
>> You need to replace localhost with the hostname of the node running
>> zookeeper. If that zookeeper is configured to use a port different than
>> 2181, then that needs to be set as well. If you have multiple zookeepers in
>> the quorum, you then zk.connect should be a comma separated list of the
>> host:port of each node.
>>
>> The default, localhost setting will only work when a drillbit is running
>> on the same node as the zookeeper.
>>
>>
>> On Sun, Oct 27, 2013 at 2:57 PM, Michael Hausenblas <
>> [EMAIL PROTECTED]> wrote:
>>
>>>
>>>> One thing to add to the diagram is that all of the drill java processes
>>> will look at what is in drill-override.conf.
>>>
>>> Thanks, done.
>>>
>>>
>>>> You must set zk.connect to the correct zk host:port.
>>>
>>>
>>> Can you be a tad more explicit, please? In drill-override.conf I have
>>>
>>> [[
>>> …
>>> zk: {
>>>        connect: "localhost:2181”,
>>> …
>>> ]]
>>>
>>>
>>> What am I overlooking?
>>>
>>> Also, any directions re the rest of my questions (re bin/submit_plan
>>> etc.)?
>>>
>>>
>>> With a little help from here,  I’m happy to put together the description
>>> how to set this up in the Wiki, also to address a query we’ve now lying
>>> around for more than three weeks, by Steve McPherson – see
>>> http://mail-archives.apache.org/mod_mbox/incubator-drill-user/201310.mbox/%3CCE71A20F.14F5B%25stevemp%40amazon.com%3E– the fact that it attracted 0 responses I find slightly embarrassing, and
>>> if I were Steve, I’d prolly not touch Drill anymore, but let’s hope for the
>>> best …
>>>
>>>
>>> Cheers,
>>>                Michael
>>>
>>> --
>>> Michael Hausenblas
>>> Ireland, Europe
>>> http://mhausenblas.info/
>>>
>>>> On 27 Oct 2013, at 21:32, Steven Phillips <[EMAIL PROTECTED]> wrote:
>>>>
>>>> One thing to add to the diagram is that all of the drill java processes
>>>> will look at what is in drill-override.conf. You must set zk.connect to
>>> the
>>>> correct zk host:port.
>>>>
>>>>
>>>> On Sun, Oct 27, 2013 at 2:00 PM, Michael Hausenblas <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>>
>>>>> Folks,
>>>>>
>>>>> I’m trying to set up Drill in distributed mode. Here’s what I have so
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB