Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Build performances


+
Jean-Marc Spaggiari 2013-02-25, 18:42
+
Andrew Purtell 2013-02-25, 18:52
Copy link to this message
-
Re: Build performances
Jean-Marc Spaggiari 2013-02-25, 19:09
Hi Andy.

With DEBUG enabled, what seems to be slow is that:
[DEBUG] Adding artifact: org.apache.commons:commons-math:jar:2.1 with
file: /home/jmspaggi/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar
to assembly location: lib/commons-math-2.1.jar.
[DEBUG] Adding file:
/home/jmspaggi/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar
to archive location: hbase-0.94.6-SNAPSHOT/lib/commons-math-2.1.jar
[DEBUG] Adding artifact: commons-net:commons-net:jar:1.4.1 with file:
/home/jmspaggi/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar
to assembly location: lib/commons-net-1.4.1.jar.
[DEBUG] Adding file:
/home/jmspaggi/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar
to archive location: hbase-0.94.6-SNAPSHOT/lib/commons-net-1.4.1.jar
About 3 minutes, displaying one line every 1 to 2 seconds.

Then this is taking a lot of time:
[INFO] Copying files to /tmp/hbase-0.94/target/hbase-0.94.6-SNAPSHOT

It's now running for 10 minutes and seems it's doing nothing. This is
what I saw without the debug enabled. I'm building on an SSD drive.
It's only about 250MB. I can even build it in memory if required. But
I don't think it's because of the disk.

JM

2013/2/25 Andrew Purtell <[EMAIL PROTECTED]>:
> If you run Maven with debug logging enabled, do you see it taking ~3
> seconds per dependency jar to decide if it wants to include it in the
> assembly or not?
>
>
>
> On Mon, Feb 25, 2013 at 10:42 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Buliding HBase locally is taking 1h for me:
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD SUCCESS
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 59:36.659s
>> [INFO] Finished at: Mon Feb 25 13:11:36 EST 2013
>> [INFO] Final Memory: 64M/593M
>> [INFO]
>> ------------------------------------------------------------------------
>>
>>
>> But from that our, it seems to be spending most of the time doing that:
>> [INFO] --- maven-assembly-plugin:2.2-beta-5:single (tarball) @ hbase ---
>> [INFO] Reading assembly descriptor: src/assembly/all.xml
>> [INFO] Copying files to
>> /home/jmspaggi/workspace/hbase-0.94/target/hbase-0.94.6-SNAPSHOT
>>
>> My CPUs are not loaded at all (1.44/8 load5), CPUWIO is almost 0 too,
>> and disks are almost not working.
>>
>> I have started the build with mvn clean site install assembly:assembly
>> -DskipTests -Prelease.
>>
>> Is that really spending all this time copŷing files? I will move the
>> build to an SSD drive and retry, but is there anyway to reduce the
>> build time?
>>
>> Thanks,
>>
>> JM
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
+
Stack 2013-02-25, 19:59
+
Jean-Marc Spaggiari 2013-02-25, 20:09
+
Stack 2013-02-25, 20:42
+
Jean-Marc Spaggiari 2013-02-25, 20:54
+
Stack 2013-02-26, 00:10
+
Jean-Marc Spaggiari 2013-02-26, 02:22
+
Jean-Marc Spaggiari 2013-02-26, 13:36
+
Stack 2013-02-26, 20:04