Jason's Maven 3 Presentation

Maven 3.0Jason van Zyl

SONATYPE

Thursday, March 19th, 2009

Monday, April 20, 2009



XML


XML


/** * This is a method of a Hibernate ORM object which can be serialized via JAXB. * Also utilizes Hibernate validation, an i18n Message manager and EqualsMember, * which means this element is a member of the set of fields denoting object * equality. Somehow, I'm starting to miss XML... * @return related details for this person */ @ManyToOne( fetch = FetchType.EAGER, cascade = { javax.persistence.CascadeType.PERSIST, javax.persistence.CascadeType.MERGE } ) @JoinColumn( name = "PERSON_DETAIL", nullable = false ) @Where( clause = "ACTV_ID = 'T'" ) @Cascade( { CascadeType.SAVE_UPDATE, CascadeType.MERGE, CascadeType.DELETE } ) @Cache( usage = CacheConcurrencyStrategy.NONSTRICT_READ_WRITE ) @EqualsMember @NotNull @Valid @Message( "${msg.invalid.person.detail}" ) @XmlElement( name = "person‐detail", required = true ) public PersonDetail getPersonDetail() { return personDetail; }


I love Maven! (or I won’t get fed)

I love Maven too! Can I have lunch now?


I’m going to try Gradle whether you like it or not.



Why Maven 3.0Maven 2.0

Components lacked degree encapsulation and layering

Code base in many places is very difficult to understand and therefore very hard to get more people involved

Fundamental problems with artifact resolution system

Maven 3.0Fewer modules, code has been reduced, and has been heavily refactored

We hope the code base will be simpler so that more people can get involved

Better support for IDEs: all Maven integration in IDEs is actually using 3.x

Model Builder Specification: detailed instruction on how the POM is constructed

Complete rewrite of the artifact resolution system



Brief HistoryOriginal Maven 2.1.x moved to Maven 3.x (trunk)

Maven 2.0.x work that involved new features became Maven 2.1.x

Three active branches Maven 2.0.x, 2.1.x and 3.0.x

I will never talk about delivery dates again, but Maven 3.x should soon be viable a replacement for 2.x.x and the integration tests are telling us so. I’m sure we’re going to run into many gnarly details but the code is getting much easier to work with and correct with each passing day. Myself, Shane Isbell, Oleg Gusakov, Benjamin Bentmann, and Igor Fedorenko are essentially working with 3.x on pretty much a full-time basis


Integration TestingAn enormous amount of time and energy has gone into improving the the integration tests for Maven. The integration tests are the gatekeeper and will let us determine that we have a Maven 3.x that is compatible for most Maven 2.x projects.


Backward CompatibilityWe must ensure that plugins and reports written against the Maven 2.0.x APIs remain viable in 3.0. We don't want people rewriting their plugins. There are several plugins that are using the current artifact resolution code that will not be supported. The ones that are in our control we can port over to use Mercury, and external users will have to deal with the major version change. Most people will not be affected and Mercury will be a far better solution.

We must also ensure that POMs of version 4.0.0 are supported in 3.x along with the behavior currently experienced. We are relying heavily on our integrations tests right now but as we move forward the work that Shane is doing on the project builder with model-builder will help us to accommodate different versions of a POM, and different formats we decide to support. The model-builder code has no limitation with respect to formats. We can support XML, or any source that anyone can dream up. These implementations may find use outside of Maven. For example someone might build something with the Maven Embedder, JRuby, and Mercury to create a JRuby-based system. The same could be done for Groovy, or Intercal.

We have managed to keep almost everything backward compatible and we will go so far as to provide an isolated execution environment that can contain older versions of the plugin API so that everything can work. We’ll see as we progress toward 3.0 GA if this is necessary. This primarily revolves around plugins that require the old artifact resolution code which is not compatible with the new artifact resolution code at the API level. We tried to preserve behavior but there’s not much we could do about the APIs.


MavenThe best is yet to come but we really needed to get the house in order first

Refactored execution configuration

Refactored project builder

Refactored plugin manager

Refactored lifecycle executor

Error & integrity reporting

Mercury

Maven embedder

Plexus/XBean (what are those?)


Refactored execution configurationRemove Settings from the core and make it a user facing configuration

Have one configuration model for request

Have one configuration model for session: session takes the request in the constructor and delegates


Refactored project builderHold unmutated (no-interpolation or inheritance) model.

Specified the rules of construction and have much better unit/IT coverage.

Have ways of transforming between model types

Significant cleanup of profiles (uses same build processors)

Easier to extend poms

Made mercury integration easier

ResultsVersionless parent elements

Better version delegation

Mixins

XML POM format using attributes

Scripted POM formats (pom.{rb|groovy|py})


Basic Module Layoutmaven-project contains the original APIs for MavenProjectBuilder that core uses

maven-project-builder implements maven specific services of model-builder

model-builder is general model API


Refactored plugin managerGeneralizing and extracting the plugin manager capabilities and making it available as a library

Download artifacts from artifact repositories

Create isolated classloaders based on the artifacts downloaded

XStream-like configuration mechanism

Create isolated execution environments

We will use these features to make sure we can support all previous versions of Maven plugins

Expression evaluator will become part of the execution environment


A plugin extension point must have access to everything in the execution contextExecutionRequest

MavenSession

Project and its lineage

Plugin authors would provide known sites of extension. There may be any number of extensions points depending on what you're doing:

For the OSGi example we need an explicit place to modify manifest entries

Adding license information could just be something done at an extension point of the packaging plugins

Manipulation of web.xml in a WAR

Similar to Eclipse extension points, but essentially we would be injecting a Map of components with the same role

We need specification about ordering and guidelines where extension points could be manipulating the same resources

Plugin extension points


http://web.xml

http://web.xml

Plugin APISymmetric parameters: input and output

Annotations can be used and the javadocs tags will be deprecated

Easier to support scripted plugins, though tools like GMaven are taking the bytecode/compiler approach.

Complete separation of reports from the Plugin API


Lifecycle extension pointsClean extension points for processing at the beginning and end of a lifecycle.

Aggregator mojos bound to the lifecycle have been deprecated. This practice can produce some very strange results, and isn't really the right solution for many of the problems it attempts to solve. I'm hoping to include some better options for bracketing the normal build - both before, and after, explicitly - to make aggregator mojos obsolete, but for now they've been deprecated to avoid disrupting backward compatibility.

We just need a simple way to do some processing at the beginning and end of the lifecycle and much complexity will be reduced in the core.


Queryable lifecycleThe most important change in the embedding environment. You can actually query Maven for the complete execution before it happens.

Know about dependencies before executing will also be possible because the new artifact resolution code completely separates the metadata resolution from the artifact resolution.

Symmetric plugin parameters will be necessary to know how the model might change during execution. We need to know before hand where a plugin will generate a source directory versus having to execute the lifecycle and examine the compile source roots. This also applies to plugins that would add resources to a project.


Error & integrity reportingMuch improved error reporting where we will try to provide links to each identifiable problem we know of

Don't allow builds where versions come from non-project sources like local settings and CLI parameters

Don't allow builds where versions come from profiles that have to be activated manually


MercuryNew repository and transport layer

Developed by Greg, Jan, and Jesse from Webtide (the Jetty people)

Super fast transport -- async client with connection pooling and parallelization

Atomic downloads and deployments (with Nexus)

Full SSL support

Full PGP support

WebDAV client built-in

Full support for ranges using a pseudo boolean solver -- SAT4J

Designed, implemented and tested without POMs. Mercury can be embedded and will allow any domain specific application to retrieve, and deploy artifacts from any type of Repository: GEMs, P2, and Maven repositories are possible


EmbedderThe embedder is currently being used in all the major IDEs

m2eclipse

IDEA

Netbeans

Maven can now be integrated in any build or development tools


Plexus/XBeanPlexus has a long but quiet history. Started as an offshoot of the Apache Avalon project

Started at approximately the same time as Spring but didn’t yet support many features Maven needed

XBean is a new DI framework used to provide all the EE injection in OpenEJB and Geronimo. Annotations are now supported in Plexus thanks to XBean.

Plexus will leverage XBean to enable all forms of DI currently known

Integration with OSGi


POM ElementsMixins: adding capabilities or sets of capabilities very easily. Composition over inheritance. For example you can bring in complete release management capabilities with a mixin. The Maven project could encapsulate its release tooling as a mixin so that people could easily reuse that as a capability.

Tags and categorization in the POM

DependenciesExclude all

Properties

Parent versions not required

General POM extension is possible but still not sure of the implications if we allow this.


Customizable ComponentsUsing a directory and specifying it in the Classworlds configuration. Tycho simply has a special set of components that load first before the standard maven components and they override the standard Maven components. Here's the example based on what Tycho is currently doing which allows custom components to be used.

The embedder has the ContainerCustomizer which allow you to inject new component descriptors. This is used in the IDE integration (m2ecipse, Netbeans) for adding custom artifact resolvers.

What we ultimately need for Tycho is a way to dynamically pull in a set of components based on the packaging of a project. In our case with Tycho the packaging is maven-osgi-bundle and that should kick in the set of components that do builds for OSGi bundles. We also have another use case in Tycho where we are building OSGi bundles without a POM and actually using a manifest. In this case we need to somehow detect the manifest and then have the custom set of components kick in. In the case of Tycho we need a different project builder, and artifact resolver.


Dropping in component JARsmain is org.apache.maven.cli.MavenCli from plexus.core set maven.home default ${user.home}/m2 [plexus.core] load ${maven.home}/tycho/*.jar load ${maven.home}/lib/*.jar


Using the Container Customizerpublic void customize( PlexusContainer container ) { ComponentDescriptor resolverDescriptor = container.getComponentDescriptor( ArtifactResolver.class ); resolverDescriptor.setImplementation( EclipseArtifactResolver.class );}


ToolchainsPlugin denotes what toolchain type it requires for it's operation. So compilation, surefire, jnlp, ... need a JDK toolchain instance for example. The actual instance of the toolchain is passed into the plugin by the infrastructure (using MavenSession). Most current plugins that use JDK for processing, use the maven's JDK instance by default. The only change introduced is to use the toolchain in the MavenSession if found. If not, do as we did so far.

User defines the toolchain instances that are available in his current setup. Shall be user based, project independent, stored in $HOME/.m2/toolchains.xml file.

Project shall be allowed to state which instance of the given toolchain type it requires for the build. Therefore making sure that all plugins use jdk15 for example. if such toolchain instance is not found in user's local setup, the build shall fail early. For this purpose a new plugin (maven-toolchain-plugin) is introduced. It is responsible for matching the correct toolchain instances from the user's setup against the requirements of the project and make it available for other plugins to use (in the build context).


Toolchains<toolchains> <toolchain> <type>jdk</type> <provides> <version>1.5</version> <vendor>sun</vendor> <id>for_mevenide</id> </provides> <configuration> <jdkHome>/home/mkleint/javatools/jdk</jdkHome> </configuration> </toolchain> <toolchain> <type>jdk</type> <provides> <version>1.6.0</version> </provides> <configuration> <jdkHome>/home/mkleint/javatools/jdk1.6.0</jdkHome> </configuration> </toolchain>


Toolchain example /** @component */ private ToolchainManager toolchainManager;

/** * @parameter expression="${session}" * @required * @readonly */ private MavenSession session;

public void execute() { ... //get toolchain from context Toolchain tc = toolchainManager.getToolchainFromBuildContext( "jdk", session ); if ( tc != null ) { getLog().info( "Toolchain in javadoc-plugin: " + tc ); //when the executable to use is explicitly set by user in mojo's parameter, ignore toolchains. if ( javadocExecutable != null) { getLog().warn( "Toolchains are ignored, 'javadocExecutable' parameter is set to " + javadocExecutable ); } else { //assign the path to executable from toolchains javadocExecutable = tc.findTool( "javadoc" ); //NOI18N } } ... }


Incremental build supportWe needed better integration with m2eclipse. It is possible to hack around Maven core and we have done this right now as part of this experiment, but it needs to be fixed correctly in Maven.

So far we have only modified three plugins so, but it seems to be working work.

We have modified:maven-resources-plugin: generated resources

modello-maven-plugin: generated sources

plexus-component-metadata: annotation processing producing resources


Extensible reportingCreate a completely separate system for reporting. It should not be part of Maven’s core and should really live in its own world. Different reporting solutions should be supported: not just the maven-site-plugin solution which is completely coupled to Doxia. Maven 3.x will not be coupled to Doxia. An entirely separate execution environment is required for reporting.

A data production model for build information is more useful then the document production model. This is where we could completely mesh with Hudson so that Maven reports could be used to produce useful trending information. This approach would also be useful for quality reporting systems like Sonar.


Maven4Rclass Maven def initialize goals @goals = goals end

include_class 'java.io.File' include_class 'org.apache.maven.embedder.MavenEmbedder' include_class 'org.apache.maven.embedder.DefaultConfiguration' include_class 'org.apache.maven.execution.DefaultMavenExecutionRequest'

def run configuration = DefaultConfiguration.new maven = MavenEmbedder.new(configuration) r = DefaultMavenExecutionRequest.new r.setGoals( @goals ) result = maven.execute( r ); endend

m = Maven.new( ["install"] ).run


Documents

Jason's Maven 3 Presentation