Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Drill >> mail # user >> Distributed mode troubles: ZK/Curator connection time out


+
Michael Hausenblas 2013-10-27, 21:00
+
Steven Phillips 2013-10-27, 21:32
+
Michael Hausenblas 2013-10-27, 21:57
+
Steven Phillips 2013-10-27, 22:17
+
Michael Hausenblas 2013-10-27, 22:39
+
Steven Phillips 2013-10-27, 22:44
+
Steven Phillips 2013-10-27, 22:48
+
Michael Hausenblas 2013-10-28, 09:42
+
Jacques Nadeau 2013-10-28, 20:15
+
Michael Hausenblas 2013-10-28, 09:26
+
Steven Phillips 2013-10-27, 22:35
Copy link to this message
-
Re: Distributed mode troubles: ZK/Curator connection time out
Perfect, thank you very much Steven! Will give it a try first thing tomorrow morning.

Cheers,
             Michael

Sent from my iPad

--
Michael Hausenblas, http://mhausenblas.info

> On 27 Oct 2013, at 22:35, Steven Phillips <[EMAIL PROTECTED]> wrote:
>
> As for the submit_plan tool, unfortunately at the time of the M1 release,
> the tool was not fully baked. So it does not have very good parsing
> capabilities. But as you wrote in your diagram, it takes three parameters,
> a file, a type, and a zk connect string. The file is a path on the local
> filesystem to a file that contains either a logical or a physical plan, in
> json format. The type should be either physical or logical, and this
> corresponds to the type of plan that the file parameter points to. And the
> zk connect string is the zk connect string for the cluster you want to
> submit to.
>
> for example:
>
> submit_plan /plans/plan1.json physical 1.1.1.1:5181,2.2.2.2:5181 will use
> the zk quorum of servers running on 1.1.1.1 and 2.2.2.2 to find and connect
> to a drillbit, and then submit the physical plan contained in
> /plans/plan1.json to the drillbit. It will then print results to the screen
> as they come in.
>
> If you build drill off of master, the submit_plan tool has been fixed
> somewhat:
>
> $ bin/submit_plan -h
>
> Usage: ./submit_plan [options]
>  Options:
>    -h, -help, --help
>       show usage
>       Default: false
>    -bits
>       number of drillbits to run. local mode only
>       Default: 1
>  * -f
>       file containing plan
>    -local
>       run query in local mode
>       Default: false
>  * -t
>       type of plan, logical/physical
>    -zk
>       zookeeper connect string.
>       Default: localhost:2181
>
>
>
> On Sun, Oct 27, 2013 at 3:17 PM, Steven Phillips <[EMAIL PROTECTED]>wrote:
>
>> You need to replace localhost with the hostname of the node running
>> zookeeper. If that zookeeper is configured to use a port different than
>> 2181, then that needs to be set as well. If you have multiple zookeepers in
>> the quorum, you then zk.connect should be a comma separated list of the
>> host:port of each node.
>>
>> The default, localhost setting will only work when a drillbit is running
>> on the same node as the zookeeper.
>>
>>
>> On Sun, Oct 27, 2013 at 2:57 PM, Michael Hausenblas <
>> [EMAIL PROTECTED]> wrote:
>>
>>>
>>>> One thing to add to the diagram is that all of the drill java processes
>>> will look at what is in drill-override.conf.
>>>
>>> Thanks, done.
>>>
>>>
>>>> You must set zk.connect to the correct zk host:port.
>>>
>>>
>>> Can you be a tad more explicit, please? In drill-override.conf I have
>>>
>>> [[
>>> …
>>> zk: {
>>>        connect: "localhost:2181”,
>>> …
>>> ]]
>>>
>>>
>>> What am I overlooking?
>>>
>>> Also, any directions re the rest of my questions (re bin/submit_plan
>>> etc.)?
>>>
>>>
>>> With a little help from here,  I’m happy to put together the description
>>> how to set this up in the Wiki, also to address a query we’ve now lying
>>> around for more than three weeks, by Steve McPherson – see
>>> http://mail-archives.apache.org/mod_mbox/incubator-drill-user/201310.mbox/%3CCE71A20F.14F5B%25stevemp%40amazon.com%3E– the fact that it attracted 0 responses I find slightly embarrassing, and
>>> if I were Steve, I’d prolly not touch Drill anymore, but let’s hope for the
>>> best …
>>>
>>>
>>> Cheers,
>>>                Michael
>>>
>>> --
>>> Michael Hausenblas
>>> Ireland, Europe
>>> http://mhausenblas.info/
>>>
>>>> On 27 Oct 2013, at 21:32, Steven Phillips <[EMAIL PROTECTED]> wrote:
>>>>
>>>> One thing to add to the diagram is that all of the drill java processes
>>>> will look at what is in drill-override.conf. You must set zk.connect to
>>> the
>>>> correct zk host:port.
>>>>
>>>>
>>>> On Sun, Oct 27, 2013 at 2:00 PM, Michael Hausenblas <
>>>> [EMAIL PROTECTED]> wrote:
>>>>
>>>>>
>>>>> Folks,
>>>>>
>>>>> I’m trying to set up Drill in distributed mode. Here’s what I have so
+
Timothy Chen 2013-10-27, 21:50