SAC:Repo

OSGeo provides a https://repo.osgeo.org/ manager to support our projects in distribution of programming artifacts (see also Download Server ). The repository manager is a structured web service intended for consumption by build systems and can be configured to work with Maven, Docker, npm, and others.

Maven

How to use OSGeo Maven repository

To make use of repo.osgeo.org to access OSGeo java projects.

Example pom.xml file

  <repositories>
    <repository>
      <id>osgeo-release</id>
      <name>OSGeo Repository</name>
      <url>https://repo.osgeo.org/repository/release/</url>
      <!-- Contains releases for OSGeo projects: GeoTools, GeoWebCache, GeoServer, ...     -->
      <!-- Includes third-party-dependences, and caches of:                                -->
      <!-- ucar (https://artifacts.unidata.ucar.edu/content/repositories/unidata-releases) -->
      <!-- geosolutions (http://maven.geo-solutions.it/)                                   -->
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
  </repositories>

Note that the above URL is intended for maven, and is not directly browsable:

• https://repo.osgeo.org/repository/release/ - maven
• https://repo.osgeo.org/#browse/browse:release - browse and search contents
• https://repo.osgeo.org/service/rest/repository/browse/release/ - browse index

How to fix a broken build

If your project previously used:

• http://download.osgeo.org/webdav/geotools/
• http://repo.boundlessgeo.com/main/

Your build will fail with an error similar to:

[ERROR] Failed to execute goal on project gt-opengis: Could not resolve dependencies for project org.geotools:gt-opengis:jar:21-SNAPSHOT: The following artifacts could not be resolved: javax.media:jai_codec:jar:1.1.3, javax.media:jai_core:jar:1.1.3: Failure to find javax.media:jai_codec:jar:1.1.3 in http://download.osgeo.org/webdav/geotools/ was cached in the local repository, resolution will not be reattempted until the update interval of osgeo has elapsed or updates are forced -> [Help 

Please update your build, replacing:

<repository>
    <id>osgeo</id>
    <name>Open Source Geospatial Foundation Repository</name>
    <url>http://download.osgeo.org/webdav/geotools/</url>
</repository>

With the following:

<repositories>
  <repository>
    <id>osgeo</id>
    <name>OSGeo Release Repository</name>
    <url>https://repo.osgeo.org/repository/release/</url>
    <snapshots><enabled>false</enabled></snapshots>
    <releases><enabled>true</enabled></releases>
  </repository>
</repositories>

Alternative: Update settings.xml to mirror webdav and boundlessgeo repositories:

<mirrors>
  <mirror>
    <id>osgeo-release</id>
    <name>OSGeo Repository</name>
    <url>https://repo.osgeo.org/repository/release/</url>
    <mirrorOf>osgeo</mirrorOf>     <!-- previously http://download.osgeo.org/webdav/geotools/ -->
  </mirror>
  <mirror>
    <id>geoserver-releases</id>
    <name>Boundless Repository</name>
    <url>https://repo.osgeo.org/repository/Geoserver-releases/</url>
    <mirrorOf>boundless</mirrorOf> <!-- previously http://repo.boundlessgeo.com/main/ -->
  </mirror>
</mirrors>

For additional information, including how to make use of SNAPSHOTS, please see GeoTools User Guide Upgrade Instructions.

How to integrate with Artifactory as a remote repository

A known issue with Artifactory integration written for Nexus 2. We use Nexus 3 leading to the following workaround:

To integrate repo.osgeo.org as a remote repository into artifactory:

1. Use the nexus URL: ​https://repo.osgeo.org/repository/release/

2. Note that testing the URL before saving it will give a 404 warning

3. After running your maven build artifactory will be able to fetch the the artifacts

This problem occurs as the artifactory test expects to be able to browse to the Nexus URL location. However in Nexus 3 repository URLs are machine readable, with a separate URL for browsing.

Maven Instructions for OSGeo Projects

OSGeo Projects (and community projects) are encouraged to make use of the repo.osgeo.org for releases:

0. Before you start, you will need an OSGeo User ID

1. Create a TRAC ticket https://trac.osgeo.org/osgeo requesting repo access with the name of your project, and OSGeo UserID requiring access. A project repository (maphappy-releases) will be setup, with access granted to the OSGeo UserID requested.

2. Update your pom.xml repositories with osgeo release repository.

  <repositories>
    <repository>
      <id>osgeo-release</id>
      <name>OSGeo Repository</name>
      <url>https://repo.osgeo.org/repository/release/</url>
      <!-- Contains releases for OSGeo projects: GeoTools, GeoWebCache, GeoServer, ...     -->
      <!-- Includes third-party-dependences, and caches of:                                 -->
      <!-- ucar (https://artifacts.unidata.ucar.edu/content/repositories/unidata-releases) -->
      <!-- geosolutions (http://maven.geo-solutions.it/)                                   -->
      <snapshots><enabled>false</enabled></snapshots>
      <releases><enabled>true</enabled></releases>
    </repository>
  </repositories>

3. Update your pom.xml distribution management with project releases repository.

  <distributionManagement>
    <repository>
      <id>maphappy</id>
      <name>OSGeo GeoServer Release Repository</name>
      <url>https://repo.osgeo.org/repository/maphappy-releases/</url>
    </repository>
  </distributionManagement>

4. Update your settings.xml with OSGeo UserID credentials

    <server>
      <id>osgeo</id>
      <username>maphappy</username>
      <password>Foss4gForever</password>
    </server>

5. Advanced: Your project releases repository should be usable to synchronize your artifacts to maven central. We are looking at becoming an approved forge for maven central upload.

7. Advanced: For projects running a build system (jenkins, GitHub actions) you may wish set up a snapshot repository for your team.

    <repository>
      <id>osgeo-snapshot</id>
      <name>OSGeo Repository</name>
      <url>https://repo.osgeo.org/repository/snapshot/</url>
      <snapshots><enabled>true</enabled></snapshots>
      <releases><enabled>false</enabled></releases>
    </repository>

  <distributionManagement>
    <snapshotRepository>
      <id>maphappy</id>
      <name>OSGeo GeoServer Snapshot Repository</name>
      <url>https://repo.osgeo.org/repository/maphappy-snapshots/</url>
     </snapshotRepository>
  </distributionManagement>

If you have any questions please use your TRACK ticket for communication.

For a working example see pom.xml from the GeoWebCache project.

Docker

OSGeo provides a nexus based docker registry for Projects to use.

Current docker repositories are as follows: Group Repositories

• docker - docker.osgeo.org (this groups all the docker project repositories, and should be used just for docker pull requests)

Project hosted repositories (should be used only for pushes)

• PostGIS - postgis-docker.osgeo.org (writes to repo postgis-docker) - PostGIS docker development guide decribing PostGIS project use.
• Geos - geos-docker.osgeo.org (writes to repo geos-docker)
• Geoserver - geoserver-docker.osgeo.org (writes to repo geoserver-docker)

Note for pulling, all pulling should be done via docker.osgeo.org and that allows anoymous pull. All the projects are grouped together under the docker group. Pushing needs to be done via the project specific docker domains by first logging into the respective domains docker login <project-docker-domain> with your OSGeo LDAP account or build user login. As required by docker, in order to push an image, the repo of the image must be prefixed with the project domain e.g. postgis-docker.osgeo.org/postgis/build-test:alpine would be the name we might tag a build so we can push it to postgis-docker.osgeo.org.

An image pushed to any project docker repo can be pulled via docker.osgeo.org. Example

  docker pull docker.osgeo.org/postgis/build-test:alpine

Would pull the image pushed to postgis-docker.osgeo.org.

Docker Instructions for OSGeo Projects Teams

OSGeo Projects (and community projects) are encouraged to make use of the repo.osgeo.org for docker:

0. Before you start, you will need an OSGeo User ID

1. Create a TRAC ticket https://trac.osgeo.org/osgeo requesting repo access with the name of your project, and OSGeo UserID requiring access.

As an example see this ticket , leading to the creation of `postgis-docker` repository, with access granted to a `postgis-build` and `robe` OSGeo userids.

Nexus

The repo.osgeo.org is currently managed as an instance of Sonatype Nexus Repository Manager as shown below.

• This is a repository manager intended for use by build systems
• If you like you can use search to locate specific artifacts
• Nexus also offers a chance to browse contents with an html view
• Artifact URLs are stable and can be used as an alternative to downloads.osgeo.org and does provide download counts.

Repositories

Maven repository naming conventions:

• project-releases (hosted) - each project manages its releases in a distinct hosted repository. They may wish to upload third party dependencies here also.
• project-snapshots (hosted) - each project with build automation manages its snapshots in a distinct hosted repository, subject to a policy expiring artifacts after an appropriate time
• external-cache (proxy) - external repositories can be cached at repo.osgeo.org to improve build times. Each external repository subject to routing policies to limit contents stored.
• release (group) - Gathering all releases (from multiple project releases repositories, upstream cached repositories) providing a single repository for the community
• snapshot (group) - Gather all snapshots from osgeo projects into a single repository for the community

Docker repository naming conventions:

• project-docker (hosted) - each project manages images in a distinct hosted repository
• docker (group) - Gather all docker projects into a single repository for the community to pull from

Nexus stores artifacts in blob storage (presently on the file system). We are defining different blob stores for different kinds of content:

• releases - osgeo projects artifacts
• docker - policy will be defined in conjunction with initial projects making use of this service
• snapshots - publish transit artifacts produced from the active development branches (See automation below for cleanup policies and tasks)
• cache - isolated into separate storage so we have the option of avoid backups in the future

Automation:

• snapshot_policy (cleanup policy) - unused as we have switched to using tasks below. We experimented with defining cleanup policies to manage snapshots. Setting a limit of five days removed artifacts on maintenance branches that are updated infrequently; setting a "last downloaded before 15 days" results in too much storage used by active branches.
• hourly: clean up repositories (task) - run hourly to apply cleanup policies to our repositories
• hourly: maven delete snapshot (task) - set to clean up snapshots after two days, while maintaining a minimum of 1 snapshot
• daily: publish maven indexer files (task) - used to publish an index used by build tools to synchronize thier understanding of the artifacts we publish
• daily: delete incomplete docker uploads (task)

Routing policies:

• Routing policies have been defined to minimize the amount of artifacts cached from upstream repositories

Users and groups

We anticipate a very small number of users with access to this system, primarily projects steering committee members, and artificial users created for individual build processes.

• Users are added via LDAP so your OSGeo User ID is required
• Build processes: User is created locally on the Nexus instances with access to a project specific repository.
• Groups, while LDAP groups are available the small number of users has not warranted making use of this facility