Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - hive jobs pending so long


Copy link to this message
-
Re: hive jobs pending so long
Nitin Pawar 2012-12-19, 18:58
does not look anything odd in here

do you see any task failures in your job on the jobtracker UI ?
On Wed, Dec 19, 2012 at 11:45 PM, Neil Guo <[EMAIL PROTECTED]> wrote:

>
> Yes,  hadoop replication factor is already set to 1
>
> Space on the datenode,
>
> *11291 files and directories, 39205 blocks = 50496 total. Heap Size is
> 858.94 MB / 2.6 GB (32%)
> *
>  Configured Capacity : 9.92 TB DFS Used : 2.12 TB Non DFS Used : 6.35 TBDFS Remaining:1.45 TBDFS Used%:21.39 %DFS Remaining%:14.58 %Live Nodes
> : 1 Dead Nodes : 0 Decommissioning Nodes : 0 Number of Under-Replicated
> Blocks : 164
> Result of jmap NN,
>
> Attaching to process ID 15949, please wait...
> Debugger attached successfully.
> Server compiler detected.
> JVM version is 19.0-b09
>
> using thread-local object allocation.
> Parallel GC with 13 thread(s)
>
> Heap Configuration:
>    MinHeapFreeRatio = 40
>    MaxHeapFreeRatio = 70
>    MaxHeapSize      = 3145728000 (3000.0MB)
>    NewSize          = 1310720 (1.25MB)
>    MaxNewSize       = 17592186044415 MB
>    OldSize          = 5439488 (5.1875MB)
>    NewRatio         = 2
>    SurvivorRatio    = 8
>    PermSize         = 21757952 (20.75MB)
>    MaxPermSize      = 174063616 (166.0MB)
>
> Heap Usage:
> PS Young Generation
> Eden Space:
>    capacity = 527171584 (502.75MB)
>    used     = 138223192 (131.81990814208984MB)
>    free     = 388948392 (370.93009185791016MB)
>    26.219772877591218% used
> From Space:
>    capacity = 21954560 (20.9375MB)
>    used     = 8190336 (7.8109130859375MB)
>    free     = 13764224 (13.1265869140625MB)
>    37.30585354477612% used
> To Space:
>    capacity = 21954560 (20.9375MB)
>    used     = 0 (0.0MB)
>    free     = 21954560 (20.9375MB)
>    0.0% used
> PS Old Generation
>    capacity = 351535104 (335.25MB)
>    used     = 33554448 (32.00001525878906MB)
>    free     = 317980656 (303.24998474121094MB)
>    9.545120136849832% used
> PS Perm Generation
>    capacity = 21757952 (20.75MB)
>    used     = 20201176 (19.265342712402344MB)
>    free     = 1556776 (1.4846572875976562MB)
>    92.8450251200113% used
>
>
> Resutes of JT,
>
> Attaching to process ID 16342, please wait...
> Debugger attached successfully.
> Server compiler detected.
> JVM version is 19.0-b09
>
> using thread-local object allocation.
> Parallel GC with 13 thread(s)
>
> Heap Configuration:
>    MinHeapFreeRatio = 40
>    MaxHeapFreeRatio = 70
>    MaxHeapSize      = 3145728000 (3000.0MB)
>    NewSize          = 1310720 (1.25MB)
>    MaxNewSize       = 17592186044415 MB
>    OldSize          = 5439488 (5.1875MB)
>    NewRatio         = 2
>    SurvivorRatio    = 8
>    PermSize         = 21757952 (20.75MB)
>    MaxPermSize      = 174063616 (166.0MB)
>
> Heap Usage:
> PS Young Generation
> Eden Space:
>    capacity = 131792896 (125.6875MB)
>    used     = 72740936 (69.37116241455078MB)
>    free     = 59051960 (56.31633758544922MB)
>    55.1933664163507% used
> From Space:
>    capacity = 21954560 (20.9375MB)
>    used     = 4555392 (4.3443603515625MB)
>    free     = 17399168 (16.5931396484375MB)
>    20.749183768656717% used
> To Space:
>    capacity = 21954560 (20.9375MB)
>    used     = 0 (0.0MB)
>    free     = 21954560 (20.9375MB)
>    0.0% used
> PS Old Generation
>    capacity = 351535104 (335.25MB)
>    used     = 0 (0.0MB)
>    free     = 351535104 (335.25MB)
>    0.0% used
> PS Perm Generation
>    capacity = 24313856 (23.1875MB)
>    used     = 24162376 (23.04303741455078MB)
>    free     = 151480 (0.14446258544921875MB)
>    99.37698076356132% used
>
>
> Not very familiar with Java, if I miss something, please just let me know.
>
> Thanks.
>
>
>
>
>
>
>
> On Thu, Dec 20, 2012 at 1:47 AM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>
>> i hope hadoop replication factor is set to 1
>> apart from this can you check how much disk space is empty on the
>> datanode?
>>
>> how are the memory stats on JT and NN?
>>
>>
>>
>> On Wed, Dec 19, 2012 at 10:57 PM, Neil Guo <[EMAIL PROTECTED]> wrote:
>>
>>> No, actually there's only one datanode.
Nitin Pawar