Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Bigtop, mail # user - Probably Bugs in Hive Install


+
Steven Núñez 2013-11-21, 03:29
Copy link to this message
-
Re: Probably Bugs in Hive Install
Roman Shaposhnik 2013-11-26, 05:40
Hi Steve!

Hive metastore is currently a very sorry area for Bigtop. We did have ideas on
creating a Bigtop-centric Hive metastore package based on PostgreSQL
but I haven't seen much progress there. Patches are more than welcome!

Thanks,
Roman.

On Wed, Nov 20, 2013 at 7:29 PM, Steven Núñez <[EMAIL PROTECTED]> wrote:
> Gents,
>
> The summary of this is: the system is reporting metastore not running (nor
> hive server 2), yet hive is now working. See below email where I was going
> to request diagnoses help, only to discover that the example now works, for
> reasons unknown.
>
> Cheers,
> - SteveN
>
>
> -------------------------------------- Original Message
> -----------------------------
> I’ve got a bit closer. The metastore and hive server aren’t running:
>
> org.apache.thrift.transport.TTransportException: Could not create
> ServerSocket on address 0.0.0.0/0.0.0.0:9083
> org.apache.thrift.transport.TTransportException: Could not create
> ServerSocket on address 0.0.0.0/0.0.0.0:10000
>
> Because something is already bound to those ports (from netstat):
>
> tcp        0      0 0.0.0.0:10000               0.0.0.0:*
> LISTEN      3071/java
> tcp        0      0 0.0.0.0:9083                0.0.0.0:*
> LISTEN      2827/java
>
> Whatever is listening on those ports doesn’t speak HTTP, and 10000 just
> closes a telnet connection straight away.
>
> Grepping through /etc doesn’t produce much of value, or anything I don’t
> already know:
>
> nunez$ grep -R 9083 *
> alternatives/hive-conf/hive-site.xml:
> <value>thrift://localhost:9083</value>
> grep: alternatives/oozie-tomcat-conf/webapps/oozie/ext-2.2: No such file or
> directory
> default/hive-hcatalog-server:export METASTORE_PORT=9083
> hive/conf.dist/hive-site.xml:  <value>thrift://localhost:9083</value>
> hive/conf/hive-site.xml:  <value>thrift://localhost:9083</value>
> grep: oozie/tomcat-deployment.http/webapps/oozie/ext-2.2: No such file or
> directory
> grep: oozie/tomcat-deployment.https/webapps/oozie/ext-2.2: No such file or
> directory
> grep: oozie/tomcat-deployment/webapps/oozie/ext-2.2: No such file or
> directory
>
>
> I modified the hive-site.xml file based on a ‘blog post I found earlier, but
> the port was bound even before that change.
>
> So, does anyone have any ideas about who is listening to these ports and
> why? I would have thought them reserved for Hive, but some Java program
> thinks otherwise. This of course is assuming that my Hive failure is caused
> by something related:
>
> STOP — THIS IS STRANGE.
>
> So, what was going to appear here is the Hive failure I reported yesterday.
> However upon repeating the commands, it now works:
>
> hive> create table doh(id int);
> OK
> Time taken: 5.344 seconds
>
>
> So, I guess the question now is: Why is this working when ‘service
> —status-all’ reports that the services are failed, and there’s the above two
> socket errors in the log files?
>
>
> Cheers,
> - SteveN
>
> P.S. Interesting that grep is reporting some kind of ooze filesystem errors.
> Oozie is also failing, but I haven’t got around to looking at it yet. I
> wonder if these errors are part of the reason though.
+
Mark Grover 2013-11-26, 18:33