SAC:Repo

From OSGeo
Jump to navigation Jump to search

OSGeo provides a https://repo.osgeo.org/ manager to support our projects in distribution of programming artifacts (see also Download Server ). The repository manager is a structured web service intended for consumption by build systems and can be configured to work with Maven, Docker, npm, and others.

Maven

How to use OSGeo Maven repository

To make use of repo.osgeo.org to access OSGeo java projects.

Example pom.xml file 

  <repositories>
    <repository>
      <id>osgeo-release</id>
      <name>OSGeo Repository</name>
      <url>https://repo.osgeo.org/repository/release/</url>
      <!-- Contains releases for OSGeo projects: GeoTools, GeoWebCache, GeoServer, ...     -->
      <!-- Includes third-party-dependences, and caches of:                                -->
      <!-- ucar (https://artifacts.unidata.ucar.edu/content/repositories/unidata-releases) -->
      <!-- geosolutions (http://maven.geo-solutions.it/)                                   -->
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
  </repositories>

How to fix a broken build

If your project previous used:

Your build will fail with an error similar to: 

[ERROR] [FATAL] Non-parseable POM /Users/gsbuild/.m2/repository/org/codehaus/mojo/build-helper-maven-plugin/3.0.0/build-helper-maven-plugin-3.0.0.pom: end tag name </head> must be the same as start tag <meta> from line 17 (position: TEXT seen ...//repo.osgeo.org/static/css/nexus-content.css?3.22.0-02"/>\n</head>... @20:8)  @ line 20, column 8

Please update your build, replacing: 

<repository>
    <id>osgeo</id>
    <name>Open Source Geospatial Foundation Repository</name>
    <url>http://download.osgeo.org/webdav/geotools/</url>
</repository>

With the following: 

<repositories>
  <repository>
    <id>osgeo</id>
    <name>OSGeo Release Repository</name>
    <url>https://repo.osgeo.org/repository/release/</url>
    <snapshots><enabled>false</enabled></snapshots>
    <releases><enabled>true</enabled></releases>
  </repository>
</repositories>

Alternative: Update settings.xml to mirror webdav and boundlessgeo repositories: 

<mirrors>
  <mirror>
    <id>osgeo-release</id>
    <name>OSGeo Repository</name>
    <url>https://repo.osgeo.org/repository/geotools-releases/</url>
    <mirrorOf>osgeo</mirrorOf>     <!-- previously http://download.osgeo.org/webdav/geotools/ -->
  </mirror>
  <mirror>
    <id>geoserver-releases</id>
    <name>Boundless Repository</name>
    <url>https://repo.osgeo.org/repository/Geoserver-releases/</url>
    <mirrorOf>boundless</mirrorOf> <!-- previously http://repo.boundlessgeo.com/main/ -->
  </mirror>
</mirrors>


For additional information, including how to make use of SNAPSHOTS, please see GeoTools User Guide Upgrade Instructions.

Maven Instructions for OSGeo Projects

OSGeo Projects (and community projects) are encouraged to make use of the repo.osgeo.org for releases:

0. Before you start, you will need an OSGeo User ID

1. Create a TRAC ticket https://trac.osgeo.org/osgeo requesting repo access with the name of your project, and OSGeo UserID requiring access. 

  A project repository (maphappy-releases) will be setup.

2. Update your pom.xml repositories with osgeo release repository. 

  <repositories>
    <repository>
      <id>osgeo-release</id>
      <name>OSGeo Repository</name>
      <url>https://repo.osgeo.org/repository/release/</url>
      <!-- Contains releases for OSGeo projects: GeoTools, GeoWebCache, GeoServer, ...     -->
      <!-- Includes third-party-dependences, and caches of:                                 -->
      <!-- ucar (https://artifacts.unidata.ucar.edu/content/repositories/unidata-releases) -->
      <!-- geosolutions (http://maven.geo-solutions.it/)                                   -->
      <snapshots>
        <enabled>false</enabled>
      </snapshots>
      <releases>
        <enabled>true</enabled>
      </releases>
    </repository>
  </repositories>

3. Update your pom.xml distribution management with project releases repository. 

  <distributionManagement>
    <repository>
      <id>maphappy</id>
      <name>OSGeo GeoServer Release Repository</name>
      <url>https://repo.osgeo.org/repository/maphappy-releases/</url>
    </repository>
  </distributionManagement>

4. Update your settings.xml with OSGeo UserID credentials 

    <server>
      <id>osgeo</id>
      <username>maphappy</username>
      <password>Foss4gForever</password>
    </server>

5. Advanced: Your project releases repository can be used to synchronize your artifacts to maven central.

7. Advanced: For projects running a build system (jenkins, GitHub actions) you may wish set up a snapshot repository for your team. 

    <repository>
      <id>osgeo-snapshot</id>
      <name>OSGeo Repository</name>
      <url>https://repo.osgeo.org/repository/snapshot/</url>
      <snapshots>
        <enabled>true</enabled>
      </snapshots>
      <releases>
        <enabled>false</enabled>
      </releases>
    </repository>

  <distributionManagement>
    <snapshotRepository>
      <id> maphappy </id>
      <name>OSGeo GeoServer Snapshot Repository</name>
      <url>https://repo.osgeo.org/repository/maphappy-snapshots/</url>
     </snapshotRepository>
  </distributionManagement>

If you have any questions please use your TRACK ticket for communication.

For a working example see pom.xml from the GeoWebCache project.

Docker

We are seeing volunteer OSGeo Projects to test out the docker infrastructure and define best practices for the community.

  • PostGIS and Geos will be interested for our regression testing to hold images we use

Docker Instructions for OSGeo Projects Teams

OSGeo Projects (and community projects) are encouraged to make use of the repo.osgeo.org for docker:

0. Before you start, you will need an OSGeo User ID

1. Create a TRAC ticket https://trac.osgeo.org/osgeo requesting repo access with the name of your project, and OSGeo UserID requiring access. 

  As an example see this ticket , leading to the creation of `postgis-docker` repository, with access granted to a `postgis-build` and `robe` OSGeo userids.

Nexus

The repo.osgeo.org is currently managed as an instance of Sonatype Nexus Repository Manager as shown below.

  • This is a repository manager intended for use by build systems
  • If you like you can use search to locate specific artifacts
  • Nexus also offers a chance to browse contents with an html view
  • Artifact URLs are stable and can be used as an alternative to downloads.osgeo.org and does provide download counts.

Sonatype Nexus Repository Manager

Repositories

Repository naming conventions:

  • project releases (hosted) - each project manages its releases in a distinct hosted repository. They may wish to upload third party dependencies here also.
  • project snapshots (hosted) - each project with build automation manages its snapshots in a distinct hosted repository, subject to a policy expiring artifacts after an appropriate time
  • cache (proxy) - Upstream repositories can be cached at repo.osgeo.org to improve build times. Each external repository subject to routing policies to limit contents stored.
  • release (group) - Gathering all releases (from multiple project releases repositories, upstream cached repositories) providing a single repository for the community
  • snapshot (group) - Gather all snapshots from osgeo projects into a single repository for the community

Nexus stores artifacts in blob storage (presently on the file system). We are defining different blob stores for different kinds of content:

  • releases - osgeo projects artifacts
  • docker - policy will be defined in conjunction with initial projects making use of this service
  • snapshots - subject to a policy removing artifacts after an appropriate time (currently five days)
  • cache - isolated into separate storage so we have the option of avoid backups in the future

Routing policies:

  • Routing policies have been defined to minimize the amount of artifacts cached from upstream repositories

Users and groups

We anticipate a very small number of users with access to this system, primarily projects steering committee members, and artificial users created for individual build processes.

  • Users are added via LDAP so your OSGeo User ID is required
  • Build processes: User is created locally on the Nexus instances with access to a project specific repository.
  • Groups, while LDAP groups are available the small number of users has not warranted making use of this facility
Retrieved from "http://wiki.osgeo.org/w/index.php?title=SAC:Repo&oldid=123832"