Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Build performances


+
Jean-Marc Spaggiari 2013-02-25, 18:42
+
Andrew Purtell 2013-02-25, 18:52
Copy link to this message
-
Re: Build performances
Hi Andy.

With DEBUG enabled, what seems to be slow is that:
[DEBUG] Adding artifact: org.apache.commons:commons-math:jar:2.1 with
file: /home/jmspaggi/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar
to assembly location: lib/commons-math-2.1.jar.
[DEBUG] Adding file:
/home/jmspaggi/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar
to archive location: hbase-0.94.6-SNAPSHOT/lib/commons-math-2.1.jar
[DEBUG] Adding artifact: commons-net:commons-net:jar:1.4.1 with file:
/home/jmspaggi/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar
to assembly location: lib/commons-net-1.4.1.jar.
[DEBUG] Adding file:
/home/jmspaggi/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar
to archive location: hbase-0.94.6-SNAPSHOT/lib/commons-net-1.4.1.jar
About 3 minutes, displaying one line every 1 to 2 seconds.

Then this is taking a lot of time:
[INFO] Copying files to /tmp/hbase-0.94/target/hbase-0.94.6-SNAPSHOT

It's now running for 10 minutes and seems it's doing nothing. This is
what I saw without the debug enabled. I'm building on an SSD drive.
It's only about 250MB. I can even build it in memory if required. But
I don't think it's because of the disk.

JM

2013/2/25 Andrew Purtell <[EMAIL PROTECTED]>:
> If you run Maven with debug logging enabled, do you see it taking ~3
> seconds per dependency jar to decide if it wants to include it in the
> assembly or not?
>
>
>
> On Mon, Feb 25, 2013 at 10:42 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Buliding HBase locally is taking 1h for me:
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] BUILD SUCCESS
>> [INFO]
>> ------------------------------------------------------------------------
>> [INFO] Total time: 59:36.659s
>> [INFO] Finished at: Mon Feb 25 13:11:36 EST 2013
>> [INFO] Final Memory: 64M/593M
>> [INFO]
>> ------------------------------------------------------------------------
>>
>>
>> But from that our, it seems to be spending most of the time doing that:
>> [INFO] --- maven-assembly-plugin:2.2-beta-5:single (tarball) @ hbase ---
>> [INFO] Reading assembly descriptor: src/assembly/all.xml
>> [INFO] Copying files to
>> /home/jmspaggi/workspace/hbase-0.94/target/hbase-0.94.6-SNAPSHOT
>>
>> My CPUs are not loaded at all (1.44/8 load5), CPUWIO is almost 0 too,
>> and disks are almost not working.
>>
>> I have started the build with mvn clean site install assembly:assembly
>> -DskipTests -Prelease.
>>
>> Is that really spending all this time copŷing files? I will move the
>> build to an SSD drive and retry, but is there anyway to reduce the
>> build time?
>>
>> Thanks,
>>
>> JM
>>
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
+
Stack 2013-02-25, 19:59
+
Jean-Marc Spaggiari 2013-02-25, 20:09
+
Stack 2013-02-25, 20:42
+
Jean-Marc Spaggiari 2013-02-25, 20:54
+
Stack 2013-02-26, 00:10
+
Jean-Marc Spaggiari 2013-02-26, 02:22
+
Jean-Marc Spaggiari 2013-02-26, 13:36
+
Stack 2013-02-26, 20:04
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB