Difference between revisions of "Java 2018 Code Sprint"
Wiki-Jhughes (talk | contribs) (Added CCRI as an in-kind sponsor.) |
|||
(6 intermediate revisions by 3 users not shown) | |||
Line 28: | Line 28: | ||
=== Bronze Sponsors === | === Bronze Sponsors === | ||
− | [[File:AstunLogo.png|300px |link=http://astuntechnology.com]] | + | [[File:AstunLogo.png|300px |link=http://astuntechnology.com]] [[File:Osgeo-uk.png|300px |liknk=https://uk.osgeo.org/]] [[File:Atol logo.png|300px|link=https://www.atolcd.com]] |
=== In-Kind Support === | === In-Kind Support === | ||
− | [[File:Boundless_Logo.png|300px|link=http://boundlessgeo.com]] [[File:GeoCat.png|300px|link=https://www.geocat.net]] [[File:astun.png|300px|link=https://astuntechnology.com/]] | + | [[File:Boundless_Logo.png|300px|link=http://boundlessgeo.com]] [[File:GeoCat.png|300px|link=https://www.geocat.net]] [[File:astun.png|300px|link=https://astuntechnology.com/]] [[File:Geosolutions.png|150px|link=https://www.geo-solutions.it/]] |
+ | [[File:Ccri.png|150px|link=https://www.ccri.com/]] | ||
== Participants == | == Participants == | ||
Line 206: | Line 207: | ||
# JDK 11: Please install OpenJDK11 | # JDK 11: Please install OpenJDK11 | ||
#* https://jdk.java.net/11 | #* https://jdk.java.net/11 | ||
− | # | + | # Fork and clone out each project and build locally (Java 8 / master branch) |
#* jaitools | #* jaitools | ||
#* jai-ext | #* jai-ext | ||
Line 218: | Line 219: | ||
#* [https://github.com/geotools/geotools/wiki/Restructure-GeoTools-into-Jigsaw-modules Restructure GeoTools into Jigsaw modules] | #* [https://github.com/geotools/geotools/wiki/Restructure-GeoTools-into-Jigsaw-modules Restructure GeoTools into Jigsaw modules] | ||
# Required reading: | # Required reading: | ||
− | #* | + | #* This wiki page, especially '''Sprint Activities''', below. |
#* [http://openjdk.java.net/projects/jigsaw/spec/sotms The State of the Module System] / [https://www.oracle.com/corporate/features/understanding-java-9-modules.html Understanding Java 9 modules] | #* [http://openjdk.java.net/projects/jigsaw/spec/sotms The State of the Module System] / [https://www.oracle.com/corporate/features/understanding-java-9-modules.html Understanding Java 9 modules] | ||
# Recommended reading: | # Recommended reading: |
Latest revision as of 09:27, 26 October 2018
The Java tribe has a real challenge for 2018 - updating our open source projects for the Java roadmap.
Location
Sprint scheduled for Q4 2018, proposed date:
- October 22-26th
We are looking at setting up a distributed sprint with locations in:
- North America - Victoria, Boundless Office, billeting options available for those travelling
- Europe
- Software Engineer and Computer Science School, University of Sevilla, Spain
- Astun Office in Epsom, UK several point are gathering there
Contacts
Jody Garnett
Sponsors
We would like to thank our sponsors!
Silver Sponsors
Bronze Sponsors
In-Kind Support
Participants
Please add your name and the projects you are planning to sprint and note the likehood of your attendance.
# | Participant | Country | Organization | Arrival | Departure | Project Work on | Notes | Attendance |
1 | María Arias de Reyna | Spain | GeoCat bv | local | local | GeoNetwork | Notes | Confirmed |
2 | Jody Garnett | Canada | Boundless | local | local | GeoTools | Able to host guests | Confirmed |
3 | Torben Barsballe | Canada | Boundless | local | local | GT/GS | Confirmed | |
4 | Kevin Smith | Canada | Boundless | local | local | GT/GWC/GS | Confirmed | |
5 | Devon Tucker | Canada | Boundless | local | local | GT/GS | Confirmed | |
6 | Antonello Andrea | Italy | HydroloGIS | TBD | TBD | GT/JTS, Hortonmachine | Confirmed | |
7 | Ian Turton | UK | Astun | local | local | GT/GS | Epsom | |
8 | Andrea Aime | Italy | GeoSolutions | TBD | TBD | GT/GWC/GS/jai-ext/imageio-ext/jaitools | Home | Confirmed |
9 | Jim Hughes | US | CCRi | TBD | TBD | JTS/GT/GS/GeoMesa | Confirmed | |
10 | Landon Blake | US | BKF Engineers | Working Remotely | Working Remotely | JTS/Proj4J | Confirmed | |
11 | David Vick | US | Boundless | STL | remote | GT/GS | Confirmed | |
12 | Wes Richardet | US | Boundless | STL | remote | GT/GS | Confirmed |
Discussion
Java has a roadmap?
Yes, we had a previous sprint focused on Java 8 compatibility, several features had changed breaking compatibility.
More importantly the Java roadmap has changed to a six-month release cycle:
- six month release cycle is already well underway having started with Java 9 and Java 10.
- Oracle is offering three year LTS releases commercially
- AdoptOpenJDK is setting up LTS releases of OpenJDK .. backed by IBM, Microsoft and others.
- RedHat is focusing on LTS (skipping Java 9 and Java 10) and plans to ship OpenJDK based on Java SE 11
Why is updating our open source projects for the Java roadmap a challenge?
- Java changed the service provider interface plugin system used by GeoTools, forcing the project to write its own replacement.
- The java runtime has been broken up into modules, not all of which are activated by default. We need to review what sections of the JRE we require and ensure they are turned on.
- Java introduced the module system "jigsaw" providing both a class-path and module-path for loading jars.
- When loaded on the module-path jars are prevented from using the same package. This breaks multi-jar projects like GeoTools library where gt-api defining interfaces, and gt-main providing implementations.
- Jigsaw also locks down aspects of Java reflection, affecting projects like Spring that make heavy use of reflection to "auto wire" GeoServer together. Spring 5 has been released and upgrading to this release will be a key focus.
- With these changes projects like GeoServer need to review of hundreds open source dependencies to determine what other libraries are broken, if an update is available or replacement can be found.
- The java web service framework (responsible for concepts like Servlet and Session) is being removed from Oracle oversight and has been setup as [Jakarta EE Software](https://jakarta.ee).
Recommended reading
- What Comes After JDK 8?
- It's time! Migrating to Java 11 - contains working example of updating spring application
- The State of the Module System / Understanding Java 9 modules
- Add an Automatic-Module-Name manifest entry - shows guava project deciding how to name their modules
- Modules in One Lesson - Good introduction to modules.
Do you have any experience running sprints?
The GeoServer team really benefited from java 2017 code sprint and is eager to repeat the success.
GeoServer Planning
Initial planning has started:
- GSIP 171 Java 18.9 Compatibility (GeoServer)
Preflight activities:
- dependency audit
Required updates:
- Spring 5 - Older versions of spring are not compatible with Java 11. Upgrading to from Spring 4 to Spring 5 does involve handling some API changes.
- HazelCast - Like Spring, HazelCast involves a lot of reflection.
Module refactor:
- Repackage GeoServer application jars to prevent conflicts at the package level.
- Resulting application can be used on either the CLASSPATH (Java 8) or MODULEPATH (Java 11)
GeoTools Planning
Planning and work is already well underway:
- GeoTools Java-9-Compatibility (GeoTools)
The first completed phase is to allow GeoTools to be used on the classpath:
- Migrate Units to JSR-363 (GeoTools)
- FactoryRegistry Refactoring for Java 9 Compatibility (GeoTools)
The sprint goal is to refactor the geotools library into modules, allowing the jars to be used on either the classpath (Java 8) or the module path (Java 11).
- Restructure GeoTools into Jigsaw modules (GeoTools)
- Core library refactored into modules
- plugins used to automatic modules to dependencies on the classpath
In the above illustration the gt-svg module is used as an automatic module publishing org.geotools.renderer.style.svg package. It acts as a bridge to the multi-jar project batik still on the classpath, completely masking the fact batik is used used to read and render svg files. The core module gt-renderer publishes select packages for use, while hiding others. It makes use of ServiceLocator to access the IconFactory implemented by gt-svg and never has direct use of the batik implementation.
For this to work the gt-svg jar has been refactored, moving the icon factory the new package org.geotools.style.svg. This was required as the package org.geotools.renderer.style was already published by gt-renderer.
We expect:
- Initial focus is on the core library, refactoring to allow jars to be used on the module path as named modules
- plugins will remain on the classpath, accessed via service locator, any conflicting packages will not be visible to client code
JTS Topology Suite Activities
Results of bonn code sprint:
- Add module names for better Java 9/Jigsaw support (JTS Topology Suite)
JTS jars can be placed on the module-path used as an automatic module, the jars have been supplied am Automatic-Module-Name using a MANIFEST.MF entry.
For an example.application using jts-core as a module add module-info:
module example.application { requires org.locationtech.jts; // jts-core }
GeoNetwork Planning
Sprint Coordination
Communication
While many of us are meeting in person, we still need to communicate across teams and timezones:
- Java 2018 Code Sprint Activities - group spreadsheet
- https://gitter.im/OSGeo/Sprint - casual communication, ask for help if you are stuck for more than ten mins
- https://jitsi.org - hangout and share screen
Coordination
Care is required to commit everything, and hand over work from one timezone to the next.
- Use of the Java 2018 Code Sprint Activities spreadsheet is key preventing two people working on the same problem.
- Ask on gitter and put your name next to an activity
- If you need to step out, or are done for the day, be sure to commit progress and let others continue!
Preflight checklist
To help get ready we have assembled a preflight checklist:
- JDK 11: Please install OpenJDK11
- Fork and clone out each project and build locally (Java 8 / master branch)
- jaitools
- jai-ext
- imageio-ext
- geotools
- geowebcache
- geoserver
- Read the proposals:
- Required reading:
- This wiki page, especially Sprint Activities, below.
- The State of the Module System / Understanding Java 9 modules
- Recommended reading:
- Core Java SE 9 for the Impatient (Chapter 15. The Java Platform Module System)
- Understanding Java 9 modules
- jpms automatic modules
- [jpms module naming http://blog.joda.org/2017/04/java-se-9-jpms-module-naming.html]
- Background:
- Modules in One Lesson - Good introduction to modules.
- It's time! Migrating to Java 11 - contains working example of updating spring application
- What Comes After JDK 8?
- Add an Automatic-Module-Name manifest entry - shows guava project deciding how to name their modules
Sprint Activities
The sprint is broken up into several stages, goal is to have something deliverable at the end of each stage. Care has been taken to identify activities that can be worked on in parallel.
Stage 1 Build and run in JDK 11
Done when being able to start and run GeoServer on JDK11 with release extensions loaded in
Everything in our stack builds and run without any flag added, off the classpath (it's ok to have warnings). This will allow us to get JDK 11 builds going.
- Compile: Andrea has made considerable progress here, to mirror you need to check out and build locally all the dependencies
- Tests: passing tests can be worked on in parallel, as long as pervious stages were able to compile without tests. This work may require updating some dependencies
- Build: package release artifacts such as installers and javadocs
Project | Compile | Test | Build |
jaitools | - | - | - |
jai-ext | x | - | - |
imageio-ext | x | - | - |
geotools | x | - | - |
geowebcache | x | - | - |
geoserver | x | - | - |
Stage 2 Reduce warnings from Dependency Analysis Tools
Done when code we have control over has addressed dependency analysis warnings (except split packages)
This cannot be automated, and is why we have so many people in the sprint!
- Resolve Warning: Address warnings from our codebase, many warnings provide clear guidance (will need to save split packages to the next stage.)
- Upgrade Dependencies: Upgrade dependencies to resolve warnings (should still compatible with Java 8 via mule-release jar or similar.)
Coordination using our spreadsheet will be key as dependency upgrades are transitive to downstream projects.
Stage 3 Module repackage
Done when split-module warnings are resolved, and geotools demo applications can run on the module path.
Add automatic module descriptors, eliminate split packages in library projects, add module-info.java only if needed.
- Repackage: Repackage each codebase as required to address-split modules
- Module App: Make sure we can run a true module app depending on the automatic modules
- Imports: Adjust imports and the like as needed in all projects, try to collect migration scripts to help others do the same.
Discussion:
- Choice between using eclipse api baseline to generate migration script, or writing sed scripts by hand.
Stage 4 org.opengis repackage
Done when references to org.opengis package are removed
Swich gt-api away from using org.opengis package, upgrade everything else to follow
- Repackage: Repackage org.opengis interfaces into appropriate org.geotools package
- EMF Model: Update emf models and regenerate
Discussion:
- Updating EMF models requires figuring out which version of eclipse was used to generate
Postsprint Activities
- Blog capturing sprint results and thanking sponsors
- We would like to issue a milestone release giving downstream projects a chance to migrate ahead of January release candidate.
How to sponsorship
Contributions will be put towards travel costs for sprint participants who would be otherwise unable to attend. Any surplus at the end of the event will be turned over to OSGeo or used for a future code sprint. We have set-up the sprint to minimize travel and accommodation costs.
Sponsors will receive the following benefits / honours:
- Your logo at the top of this page
- Mention in project communication (for example the GeoServer 2.15 release announcement)
- The ability to run on a supported java platform
- Our gratitude :)
This event provides the following sponsorship levels:
Gold | $2500 USD | ||
Silver | $1500 USD | ||
Bronze | $750 USD | ||
In-Kind | In-kind support graciously accepted |
This is an official OSGeo event, your contribution counts towards being recognized as an OSGeo Sponsors.
For more information on sponsorship, please contact Jody Garnett, Andrea Aime.
How to Sponsor
- Sponsorship is accepted through the Open Source Geospatial Foundation
- Use OSGeo PayPal link, funds are in USD marked towards "OSGeo Foundation java code sprint".
- To make alternative arrangements or request an invoice contact OSGeo Treasurer: treasurer@osgeo.org
- Once sponsorship is received the OSGeo Treasurer will contact you and the event organizers to acknowledge your sponsorship
- Please have a logo ready for your organization if you wish to be acknowledged publicly
- Your event sponsorship, at your request, can contribute towards being recognized as an OSGeo Foundation sponsor
Sponsorship Outreach
We are reaching out to organizations to see if there is interest in sponsorship:
- Prior sponsors: gaia3d, boundless, ian, geodan, how2map, fossgis, atol, geosolutions, astun
- Local sponsors: astun, vivid solutions, transient software
If there is a lot of local sponsorship we expect to drag the developers away from the task at hand and meet the local community (at a social evening or similar).
OSGeo Funding Request
The following information is requested by OSGeo Board Code_Sprint_Guidelines, the projects involved indicated that we would be planning a sprint during the 2018 budget process.
It is intended to that OSGeo be recognized as hosting this event, rather than acknowledged as a sponsor.