Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Build performances


Copy link to this message
-
Re: Build performances
Following on from Andrew, does the debug show you anything?  (Copying
individual files?)
St.Ack
On Mon, Feb 25, 2013 at 11:09 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi Andy.
>
> With DEBUG enabled, what seems to be slow is that:
> [DEBUG] Adding artifact: org.apache.commons:commons-math:jar:2.1 with
> file:
> /home/jmspaggi/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar
> to assembly location: lib/commons-math-2.1.jar.
> [DEBUG] Adding file:
>
> /home/jmspaggi/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar
> to archive location: hbase-0.94.6-SNAPSHOT/lib/commons-math-2.1.jar
> [DEBUG] Adding artifact: commons-net:commons-net:jar:1.4.1 with file:
>
> /home/jmspaggi/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar
> to assembly location: lib/commons-net-1.4.1.jar.
> [DEBUG] Adding file:
>
> /home/jmspaggi/.m2/repository/commons-net/commons-net/1.4.1/commons-net-1.4.1.jar
> to archive location: hbase-0.94.6-SNAPSHOT/lib/commons-net-1.4.1.jar
> About 3 minutes, displaying one line every 1 to 2 seconds.
>
> Then this is taking a lot of time:
> [INFO] Copying files to /tmp/hbase-0.94/target/hbase-0.94.6-SNAPSHOT
>
> It's now running for 10 minutes and seems it's doing nothing. This is
> what I saw without the debug enabled. I'm building on an SSD drive.
> It's only about 250MB. I can even build it in memory if required. But
> I don't think it's because of the disk.
>
> JM
>
>
>
> 2013/2/25 Andrew Purtell <[EMAIL PROTECTED]>:
> > If you run Maven with debug logging enabled, do you see it taking ~3
> > seconds per dependency jar to decide if it wants to include it in the
> > assembly or not?
> >
> >
> >
> > On Mon, Feb 25, 2013 at 10:42 AM, Jean-Marc Spaggiari <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Hi,
> >>
> >> Buliding HBase locally is taking 1h for me:
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] BUILD SUCCESS
> >> [INFO]
> >> ------------------------------------------------------------------------
> >> [INFO] Total time: 59:36.659s
> >> [INFO] Finished at: Mon Feb 25 13:11:36 EST 2013
> >> [INFO] Final Memory: 64M/593M
> >> [INFO]
> >> ------------------------------------------------------------------------
> >>
> >>
> >> But from that our, it seems to be spending most of the time doing that:
> >> [INFO] --- maven-assembly-plugin:2.2-beta-5:single (tarball) @ hbase ---
> >> [INFO] Reading assembly descriptor: src/assembly/all.xml
> >> [INFO] Copying files to
> >> /home/jmspaggi/workspace/hbase-0.94/target/hbase-0.94.6-SNAPSHOT
> >>
> >> My CPUs are not loaded at all (1.44/8 load5), CPUWIO is almost 0 too,
> >> and disks are almost not working.
> >>
> >> I have started the build with mvn clean site install assembly:assembly
> >> -DskipTests -Prelease.
> >>
> >> Is that really spending all this time copŷing files? I will move the
> >> build to an SSD drive and retry, but is there anyway to reduce the
> >> build time?
> >>
> >> Thanks,
> >>
> >> JM
> >>
> >
> >
> >
> > --
> > Best regards,
> >
> >    - Andy
> >
> > Problems worthy of attack prove their worth by hitting back. - Piet Hein
> > (via Tom White)
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB