Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Re: Configure Hive in Cluster


+
venkatramanan 2013-01-17, 06:54
+
Nitin Pawar 2013-01-17, 06:59
+
nagarjuna kanamarlapudi 2013-01-17, 07:02
+
venkatramanan 2013-01-17, 07:17
+
Nitin Pawar 2013-01-17, 07:26
Copy link to this message
-
Re: Configure Hive in Cluster
venkatramanan 2013-01-17, 11:53
Can you suggest me the mandatory hive parameters and clustering
configuration steps

On Thursday 17 January 2013 12:56 PM, Nitin Pawar wrote:
> looks like a very small cluster with very limited memory to run
> mapreduce jobs also number of map/reduce slots on nodes are less so at
> a time only one map is running.
>
> but still 15 min is a lot of time for 600MB memory
>
>
> On Thu, Jan 17, 2013 at 12:47 PM, venkatramanan
> <[EMAIL PROTECTED] <mailto:[EMAIL PROTECTED]>>
> wrote:
>
>     Below details are the cluster configuration
>
>     Configured Capacity         : 82.8 GB
>     DFS Used                          : 1.16 GB
>     Non DFS Used                  : 31.95 GB
>     DFS Remaining                : 49.69 GB
>     DFS Used%                      : 1.4 %
>     DFS Remaining%              : 60.01 %
>     Live Nodes <http://localhost:50070/dfsnodelist.jsp?whatNodes=LIVE>
>                           : 2
>     Dead Nodes <http://localhost:50070/dfsnodelist.jsp?whatNodes=DEAD>
>                         : 0
>     Decommissioning Nodes
>     <http://localhost:50070/dfsnodelist.jsp?whatNodes=DECOMMISSIONING>
>     : 0
>     Number of Under-Replicated Blocks : 0
>
>     My Select Query is:
>
>     "select * from tweet where Id = 810;"
>
>     This query takes 15 min to complete
>
>
>
>     On Thursday 17 January 2013 12:29 PM, Nitin Pawar wrote:
>>     how many number of nodes you have for select query?
>>     whats your select query?
>>
>>     if its just a select * from table then it does not run any
>>     mapreduce job
>>     so its just taking time to show data on your screen if you are
>>     using that query
>>
>>
>>     On Thu, Jan 17, 2013 at 12:24 PM, venkatramanan
>>     <[EMAIL PROTECTED]
>>     <mailto:[EMAIL PROTECTED]>> wrote:
>>
>>         I didnt set any hive parameters and my total table size is
>>         610 MB only
>>
>>
>>
>>         On Thursday 17 January 2013 12:11 PM, Nitin Pawar wrote:
>>>         a bit more details on size of table and select query will help
>>>         also did you set any hive parameters ?
>>>
>>>
>>>         On Thu, Jan 17, 2013 at 12:12 PM, venkatramanan
>>>         <[EMAIL PROTECTED]
>>>         <mailto:[EMAIL PROTECTED]>> wrote:
>>>
>>>             Hi All,
>>>
>>>             Am Newbie in apache hive. I have create a table and
>>>             thats points to the HDFS Folder path and its takes 15
>>>             min to execute the simple "*select*" stmt, Can anyone
>>>             suggest me for a best practices and performance
>>>             improvement on hive.
>>>
>>>             Thanks in Advance
>>>
>>>             Venkat
>>>
>>>
>>>
>>>
>>>         --
>>>         Nitin Pawar
>>
>>
>>
>>
>>     --
>>     Nitin Pawar
>
>
>
>
> --
> Nitin Pawar

+
venkatramanan 2013-01-17, 06:42
+
Nitin Pawar 2013-01-23, 07:37
+
venkatramanan 2013-01-23, 07:58
+
Nitin Pawar 2013-01-23, 08:00