Difference between revisions of "Osgeolive-jupyter"

From OSGeo
Jump to navigation Jump to search
 
(170 intermediate revisions by 2 users not shown)
Line 1: Line 1:
  
−
== Focal Notes ==
+
== Python Libraries Strategy ==
 +
 
 +
''Fresh priority python3 for the Jupyter Notebooks of 2023''
 +
 
 +
* '''GeoPandas''' (and pySAL) : <code>gpd 0.12</code>direct link to '''GEOS 3.11''', popular extension to Pandas data science; please recall that pySAL is merging into the GeoPandas project overall;  GeoPandas+pySAL use matplotlib graphics base, prefers Folium for web maps (?)
 +
 
 +
* '''Numpy''' & '''Pandas''' : <code>np 1.21.5</code>,<code>pd 1.3.5</code> fundamental  packages
 +
 
 +
* PostGIS writing done with '''pyscopg2''' <code>2.9.2</code> (low-level) or '''geoalchemy2''' <code>0.10.2</code> (ORM based on sqlalchemy)
 +
 
 +
* '''rasterio''' :  modern geoTIFF handling; sophisticated library design; not easy to build from source <code>1.3.4</code>
 +
 
 +
* '''Shapely''' & '''Fiona''' : direct link to libGEOS/ogr from python; example Notebooks in place  <code>Shapely-1.8.5</code><code> Fiona-1.8.22</code>
 +
 
 +
* '''opendatacube'''  core and Notebook client examples [https://github.com/opendatacube/datacube-core/releases/tag/1.8.7 v1.8.7 -LINK-]  '''odc-geo'''  a minimal Opendatacube setup (see below)
 +
 
 +
* '''pySTAC'''  spatial-temporal analysis standard; opendatacube collaborators  [https://github.com/stac-utils/pystac/releases/tag/v1.4.0 v1.4 -LINK-] updt v1.6
 +
 
 +
* stactools  is a collection of interfaces and utilities, TBD from odc soon [https://github.com/stactools-packages -LINK-]
 +
 
 +
* fsspec  [https://filesystem-spec.readthedocs.io/en/latest/ -LINK-]
 +
 
 +
* '''GRASS''' notebooks [https://github.com/OSGeo/grass/tree/main/doc/notebooks -LINK-] [https://gitlab.com/geoharmonizer_inea/odse-workshop-2022/-/tree/main/grass_gis/notebooks -LINK2-] [https://twitter.com/MarkusNeteler/status/1536620657844408322 -TWEET-]
 +
 
 +
* '''actinia''' [https://github.com/mundialis/actinia-jupyter/ notebooks] - STAC adaptor [https://github.com/mundialis/actinia-stac-plugin -LINK-] [https://github.com/mundialis/actinia_core updates]
 +
 
 +
* rioxarray  xarray adaptor for GDAL NetCDF  [https://corteva.github.io/rioxarray/stable/readme.html -README-]  [https://corteva.github.io/rioxarray/stable/installation.html -INSTALL-]
 +
 
 +
* '''geopython'''  OWSLib, pygeoapi, pygeometa [https://github.com/geopython/pygeometa/releases/tag/0.9.1 v0.9.1], others .. [https://github.com/orgs/geopython/repositories All -LINK-]
 +
 
 +
* '''Scikit-Learn''' :  original machine-learning library by international research team led by French math; stands the tests of time; integrated with SciPy for Fortran, BLAS, LAPACK libs; numpy for data allocation; important primitives like PCA, SVM, and KNN are optimized in practical ways, and then benchmarked against pure Fortran or C++ for comparison <code>0.23.2</code>
 +
 
 +
* '''boto''' for AWS S3, other data store transfer; can use AWS credentials (or Hetzner, or..) modern version using pip installed locally
 +
 
 +
* '''osmium'''  libosmium2-dev, libosmium2-doc, python3-pyosmium, osmium-tool : a C++ framework for working with OSM data files; osmium library bindings for Python 3 ## <code>osmium version 1.14.0;  libosmium version 2.18.0;  Supported PBF compression types: none zlib lz4;  Copyright (C) 2013-2022  Jochen Topf <jochen@topf.org></code>
 +
 
 +
* '''mapnik'''  libmapnik3.1, python3-mapnik : mapnik rendering engine for OSM maps and others <code>libmapnik.so.3.1.0</code>
 +
 
 +
* ''mapclassify'' : part of pySAL, versions must match with geopandas setup [https://pysal.org/mapclassify/ --LINK--]
 +
 
 +
*  ''lightgbm'' :  MSFT python, updt ''pip3 install --user --upgrade-strategy=only-if-needed lightgbm'' ; ''v3.3.2''
 +
 
 +
* DASK job lib [https://github.com/dask/dask/tags -LINK-]
 +
 
 +
*  '''inline''' maps ''vs'' '''slippy''' maps : cartopy, other matplotlib variants are inline static; notebook web maps need js libs e.g. ipyleaflet
 +
 
 +
* EOmaps TODO evaluate here  [https://github.com/raphaelquast/EOmaps/blob/master/setup.py -LINK-]
 +
 
 +
* georasters TODO evaluate [https://github.com/ozak/georasters -LINK]
 +
 
 +
* Canada Met - Meteorological Service of Canada Open Data User Documentation [https://eccc-msc.github.io/open-data/readme_en/ -LINK-]
 +
 
 +
* LiDAR : this is slightly painful since modern commercial GIS is flooded with this tech, delivered by advanced Western companies seeking something novel to sell. Changes to the LASzip licensing have enabled inclusion, but as of today, no Debian maintainer in place, and bas wants PDAL out at this time. [https://lists.debian.org/debian-gis/2022/02/msg00004.html -EMAIL-] [https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1009625 -EMAIL2-]  *update* '''PDAL''' via dot-deb is included in ol15
 +
 
 +
* DeepLearning "no" :  deep learning has taken over in the modern AI/ML research, for funding and hype, but not on this disk. Please note that the ML libraries on osgeolive are completely useful, accurate and performant today.. no need for a constant and fast Internet connection, nor a compute infrastructure the size of a large naval ship, or special impenetrable models that can only be built by those machines. [http://blog.light42.com/wordpress/?p=3564 -LINK-]  Serious and in-demand applications like "facial recognition" are not included here. Many real applications to remote sensing are built using DeepLearning, but those are not available on this disk. Some benefits of not including DeepLearning : no dependencies on tricky GPU libs; not suffering from fast-updates of important ML core libraries; avoid black-box training models, and their disk space; avoid software patent problems in Western Nations.
 +
 
 +
"''I feel that a common and well-known distro like this is an opportunity to build engineering cooperation, between teams that may not interact otherwise.. like the physical code sprints''"  -darkblueb in California
 +
 
 +
Example Notebooks in previous OSGeoLive  [https://git.osgeo.org/gitea/darkblueb/ol14_ipynb_misc -LINK-]
 +
 
 +
Other Resources:  Pangeo PYTHIA [https://projectpythia.org/ -LINK-]
 +
 
 +
opendatacube "Cube in a Box" [https://github.com/opendatacube/cube-in-a-box -LINK-]
 +
 
 +
--
 +
 
 +
== Jammy Notes ==
  
 
Using PPA ''ubuntugis-unstable'' & ''Ubuntu Universe'' dot-deb packages,  
 
Using PPA ''ubuntugis-unstable'' & ''Ubuntu Universe'' dot-deb packages,  
Line 6: Line 72:
  
 
<pre>
 
<pre>
−
dbb@focalk:/etc$ cat lsb-release  
+
mjam@x10jam:~$ cat /etc/lsb-release  
 
DISTRIB_ID=Ubuntu
 
DISTRIB_ID=Ubuntu
−
DISTRIB_RELEASE=20.04
+
DISTRIB_RELEASE=22.04
−
DISTRIB_CODENAME=focal
+
DISTRIB_CODENAME=jammy
−
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"
+
DISTRIB_DESCRIPTION="Ubuntu 22.04 LTS"
 +
 
 
#--
 
#--
−
  libgeos3.10.1/focal,now  3.10.1-1~focal0 amd64
+
  libgeos3.11.1   (libgeos3.11.1)
−
  libgdal-dev:amd64 (3.4.0+dfsg-1~focal0)
+
  libgdal-dev     (3.6.2+dfsg-1~jammy0)
−
  postgresql-12-postgis-3/focal,now  3.2.0+dfsg-1~focal0 amd64
+
  postgresql-14-postgis-3     (3.3.2+dfsg-1~jammy0)
−
  PGSQL="120" GEOS="3.10.1-CAPI-1.16.0" PROJ="8.2.0" LIBXML="2.9.10" LIBJSON="0.13.1"
+
  PGSQL="140" POSTGIS="3.3.2" GEOS="3.11.1-CAPI-1.17.0" PROJ="9.1.1" LIBJSON="0.15"  
 
#--
 
#--
−
</pre>
 
−
 
−
== Focal Jupyter Overview ==
 
−
 
−
<pre>
 
−
                                    Sat 19 Feb 2022 07:18:48 PM PST
 
−
$ jupyter --version
 
−
jupyter core    : 4.6.3
 
−
jupyter-notebook : 6.0.3
 
−
qtconsole        : not installed
 
−
ipython          : 7.13.0
 
−
ipykernel        : 5.2.0
 
−
jupyter client  : 6.1.2
 
−
jupyter lab      : not installed
 
−
nbconvert        : 5.6.1
 
−
ipywidgets      : 6.0.0
 
−
nbformat        : 5.0.4
 
−
traitlets        : 4.3.3
 
−
 
−
##-------------------------------
 
−
Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
 
−
pandas version: 0.25.3  <== a weak spot in this setup
 
−
matplotlib version: 3.1.2
 
−
NumPy version: 1.17.4
 
−
SciPy version: 1.3.3
 
−
IPython version: 7.13.0
 
−
scikit-learn version: 1.0.2
 
−
##-----------------------------------------
 
−
dbb@focalk:~$ ipython3
 
−
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
 
−
Type 'copyright', 'credits' or 'license' for more information
 
−
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
 
−
 
−
In [1]: import sklearn                                                                                             
 
−
In [2]: from sklearn import show_versions                                                                           
 
−
In [3]: show_versions()                                                                                             
 
  
 
System:
 
System:
−
     python: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
+
     python: 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
 
executable: /usr/bin/python3
 
executable: /usr/bin/python3
−
   machine: Linux-5.13.0-30-generic-x86_64-with-glibc2.29
+
   machine: Linux-5.15.0-25-generic-x86_64-with-glibc2.35
  
 
Python dependencies:
 
Python dependencies:
−
           pip: 20.0.2
+
           pip: 22.0.2
−
   setuptools: 45.2.0
+
   setuptools: 59.6.0
−
       sklearn: 1.0.2
+
       sklearn: 0.23.2
−
         numpy: 1.17.4
+
         numpy: 1.21.5
−
         scipy: 1.3.3
+
         scipy: 1.8.0
−
       Cython: 0.29.14
+
       Cython: None
−
       pandas: 0.25.3      <== a weak spot in this setup
+
       pandas: 1.3.5
−
   matplotlib: 3.1.2
+
   matplotlib: 3.5.1
−
       joblib: 1.1.0
+
       joblib: 0.17.0
 
threadpoolctl: 3.1.0
 
threadpoolctl: 3.1.0
  
 
Built with OpenMP: True
 
Built with OpenMP: True
−
--
+
 
−
dbb@focalk:~$ jupyter --paths
+
</pre>
 +
 
 +
=== Jupyter jammy Version ===
 +
<pre>
 +
user@osgeolive:~$ jupyter --version
 +
Selected Jupyter core packages...
 +
IPython          : 7.31.1
 +
ipykernel        : 6.7.0
 +
ipywidgets      : 6.0.0
 +
jupyter_client  : 7.1.2
 +
jupyter_core    : 4.9.1
 +
jupyter_server  : not installed
 +
jupyterlab      : not installed
 +
nbclient        : 0.5.6
 +
nbconvert        : 6.4.0
 +
nbformat        : 5.1.3
 +
notebook        : 6.4.8
 +
qtconsole        : not installed
 +
traitlets        : 5.1.1
 +
 
 +
user@osgeolive:~$ jupyter --paths
 
config:
 
config:
−
     /home/dbb/.jupyter
+
     /home/user/.jupyter
 +
    /home/user/.local/etc/jupyter
 +
    /usr/etc/jupyter
 
     /usr/local/etc/jupyter
 
     /usr/local/etc/jupyter
 
     /etc/jupyter
 
     /etc/jupyter
 
data:
 
data:
−
     /home/dbb/.local/share/jupyter
+
     /home/user/.local/share/jupyter
 
     /usr/local/share/jupyter
 
     /usr/local/share/jupyter
 
     /usr/share/jupyter
 
     /usr/share/jupyter
 
runtime:
 
runtime:
−
     /home/dbb/.local/share/jupyter/runtime
+
     /home/user/.local/share/jupyter/runtime
 +
 
 +
</pre>
 +
 
 +
 
 +
<pre>
 +
$ apt-get source python-dask-doc
 +
Reading package lists... Done
 +
Picking 'dask' as source package instead of 'python-dask-doc'
 +
NOTICE: 'dask' packaging is maintained in the 'Git' version control system at:
 +
https://salsa.debian.org/python-team/packages/dask.git
 +
Please use:
 +
git clone https://salsa.debian.org/python-team/packages/dask.git
 +
to retrieve the latest (possibly unreleased) updates to the package.
 +
Need to get 4,321 kB of source archives.
 +
Get:1 http://mirrors.wikimedia.org/ubuntu jammy/universe dask 2022.01.0+dfsg-1ubuntu1 (dsc) ...           
 +
Fetched 4,321 kB in 18s (236 kB/s)                                                                                 
 +
dpkg-source: info: extracting dask in dask-2022.01.0+dfsg
 +
dpkg-source: info: unpacking dask_2022.01.0+dfsg.orig.tar.xz
 +
dpkg-source: info: unpacking dask_2022.01.0+dfsg-1ubuntu1.debian.tar.xz
 +
dpkg-source: info: using patch list from debian/patches/series
 +
dpkg-source: info: applying pytest-futurewarning.patch
 +
dpkg-source: info: applying remove-bokeh-embeds.patch
 +
dpkg-source: info: applying use-debian-version.patch
 +
dpkg-source: info: applying reproducible-version.patch
 +
dpkg-source: info: applying Use-the-packaged-MathJax.patch
 +
dpkg-source: info: applying use-local-intersphinx.patch
 +
dpkg-source: info: applying use-local-reference-yaml.patch
 +
dpkg-source: info: applying js-yaml.patch
 +
dpkg-source: info: applying use-youtube-nocookie.patch
 +
dpkg-source: info: applying reproducible-config-autofunction.patch
 +
dpkg-source: info: applying 32bit-comatibility.patch
 +
dpkg-source: info: applying scipy1.8.patch
 +
</pre>
 +
 
 +
=== Jupyter Notebooks ===
 +
 
 +
Notes: the demo material from Actinia/GRASS, opendatacube and others use an AWS connection. We want to demo without requiring a connection. There was an idea that a video could be a local file with the demo. Enock said he can edit video, and that some High School Summer of Code students submit their project this way, but the quality is not professional. Angelos suggested that any demo in a video should also be available to actually run, if the network is working; also the GRASS Notebooks download data, and then use that as the demo, instead of doing work only in the clouds. Brian said we should move to beta1 with ordinary Jupyter and then make changes to the details of the Notebooks and python stack.
 +
 
 +
<pre>
 +
   
 +
</pre>
 +
 
 +
 
 +
 
 +
=== opendatacube ===
 +
 
 +
The '''Open Data Cube''' ('''ODC''') is an Open-Source Geospatial Data Management and Analysis Software project that helps you harness the power of Earth observation data. At its core, the ODC is a set of Python libraries and PostgreSQL database that helps you work with geospatial raster data.
 +
 
 +
Core features:
 +
------------------------------------------------------
 +
*  Catalogue large amounts of Earth observation data
 +
*  Use a Python based API for query and data access
 +
*  Perform exploratory data analysis using popular Python libraries like Xarray, Pandas, and Matplotlib
 +
 
 +
Details
 +
 
 +
Website: https://www.opendatacube.org/
 +
 
 +
Licence: Apache 2.0
 +
 
 +
Software Version: 1.8.7
 +
 
 +
Supported Platforms: Linux, Mac, Windows
 +
 
 +
API Interfaces: Python
 +
 
 +
25aug22
 +
<pre>
 +
FOSS4G  Pete Gadomski pete.gadomski at Gmail
 +
 
 +
        pystac  v1.6.1    r/w static collections, catalogs, objects
 +
pystac-client  v0.4.0    r/w stac api
 +
    stactools  v0.4.1    create,modify utils
 +
      odc-stac  v0.3.1    load stac assets into xarray
 +
 
 +
</pre>
 +
pystac version
 +
ref: https://trac.osgeo.org/osgeolive/ticket/2393
 +
 
 +
 
 +
 
 +
 
 +
31july2022
 +
 
 +
<pre>
 +
~$ DATACUBE_CONFIG_PATH=~/.config/datacube/datacube.conf datacube -v system init
 +
2022-08-01 04:26:09,231 8491 datacube INFO Running datacube command: /usr/bin/datacube -v system init
 +
Initialising database...
 +
2022-08-01 04:26:10,032 8491 datacube.drivers.postgres._core INFO Ensuring user roles.
 +
2022-08-01 04:26:10,044 8491 datacube.drivers.postgres._core INFO Creating schema.
 +
2022-08-01 04:26:10,045 8491 datacube.drivers.postgres._core INFO Creating tables.
 +
2022-08-01 04:26:10,093 8491 datacube.drivers.postgres._core INFO Creating triggers.
 +
2022-08-01 04:26:10,100 8491 datacube.drivers.postgres._core INFO Creating added column.
 +
2022-08-01 04:26:10,104 8491 datacube.drivers.postgres._core INFO Adding role grants.
 +
2022-08-01 04:26:10,106 8491 datacube.index.postgres.index INFO Adding default metadata types.
 +
Created.
 +
Checking indexes/views.
 +
2022-08-01 04:26:10,295 8491 datacube.drivers.postgres._api INFO Checking dynamic views/indexes. (rebuild views=True, indexes=False)
 +
Done.
 +
</pre>
 +
 
 +
26july2022
 +
 
 +
<pre>
 +
$ datacube product add /path/to/landsat-clip.yml
 +
</pre>
 +
 
 +
<pre>
 +
---  landsat-clip.yml
 +
name: clip_landsat
 +
description: example ortho imagery
 +
metadata_type: eo3
 +
 
 +
license: CC-BY-4.0
 +
 
 +
metadata:
 +
  product:
 +
    name: clip_landsat
 +
 
 +
storage:
 +
  crs: EPSG:32619
 +
  resolution:
 +
    longitude: 28.497418829384827
 +
    latitude: -28.503958771796388
 +
 
 +
measurements:
 +
  - name: imagery
 +
    dtype: byte
 +
    nodata: -32768.0
 +
    units: "meter"
 +
</pre>
 +
 
 +
<pre>
 +
$ rio info data/landsat/clip10.tif
 +
{"bounds": [374141.9163735037, 4597362.958980484, 450657.48593040195, 4662494.504774039], "colorinterp": ["gray"], "count": 1, "crs": "EPSG:32619", "descriptions": [null], "driver": "GTiff", "dtype": "uint8", "height": 2285, "indexes": [1], "interleave": "band", "lnglat": [-70.054716274406, 41.81639512026538], "mask_flags": [["all_valid"]], "nodata": null, "res": [28.497418829384827, 28.503958771796388], "shape": [2285, 2685], "tiled": false, "transform": [28.497418829384827, 0.0, 374141.9163735037, 0.0, -28.503958771796388, 4662494.504774039, 0.0, 0.0, 1.0], "units": [null], "width": 2685}
 +
</pre>
 +
 
 +
<pre>
 +
# UUID of the dataset
 +
id: f884df9b-4458-bbbb-aaaa-1a52a2db8a1a
 +
$schema: 'https://schemas.opendatacube.org/dataset'
 +
 
 +
# Product name
 +
product:
 +
  name: clip_landsat_example0
 +
 
 +
# Native CRS, assumed to be the same across all bands
 +
crs: "epsg:32619"
 +
 
 +
# Mapping name:str -> { shape:    Tuple[ny: int, nx: int]
 +
#                      transform: Tuple[float x 9]}
 +
# Captures image size, and geo-registration
 +
grids:
 +
    default:  # "default" grid must be present
 +
      shape: [2285, 2685]
 +
      transform: [28.497418829384827, 0.0, 374141.9163735037, 0.0, -28.503958771796388, 4662494.504774039, 0.0, 0.0, 1.0]
 +
 
 +
# Per band storage information and references into `grids`
 +
# Bands using the "default" grid should not need to reference it
 +
measurements:
 +
  red:              # Band using "default" grid should omit `grid` key
 +
    path: red.tif    # Path relative to the dataset location
 +
 
 +
# optional dataset location (useful for public datasets)
 +
#location: https://landsatonaws.com/L8/099/072/LC08_L1GT_099072_20200523_20200523_01_RT/metadata.yaml
 +
 
 +
# Dataset properties, prefer STAC standard names here
 +
# Timestamp is the only compulsory field here
 +
properties:
 +
  eo:platform: landsat-8
 +
  eo:instrument: OLI_TIRS
 +
 
 +
  # When recording time range use dtr:{start,end}_datetime
 +
  dtr:start_datetime: 2020-01-01T07:02:02.233Z
 +
  dtr:end_datetime:  2020-01-01T07:03:04.397Z
 +
 
 +
  # ODC specific "extensions"
 +
  odc:processing_datetime: 2020-02-02T08:10:00.000Z
 +
 
 +
  odc:file_format: GeoTIFF
 +
  odc:region_code: "074071"  # provider specific unique identified for the same location
 +
                              # for Landsat '{:03d}{:03d}'.format(path, row)
 +
 
 +
  dea:dataset_maturity: final # one of: final| interim| nrt (near real time)
 +
  odc:product_family: ard    # can be useful for larger installations
 +
 
 +
# Lineage only references UUIDs of direct source datasets
 +
# Mapping name:str -> [UUID]
 +
lineage: {}  # set to empty object if no lineage is defined
 +
 
 +
</pre>
 +
 
 +
25jul2022
 +
 
 +
<pre>
 +
user@osgeolive:~$ pip install --user lark-parser
 +
Collecting lark-parser
 +
  Downloading lark_parser-0.12.0-py2.py3-none-any.whl (103 kB)
 +
    ━---------------------------------━━━ 103.5/103.5 KB 352.8 kB/s eta 0:00:00
 +
Installing collected packages: lark-parser
 +
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.                     
 +
mappyfile 0.9.7 requires jsonref==0.2, but you have jsonref 0.1 which is incompatible.           
 +
Successfully installed lark-parser-0.12.0 
 +
 
 +
$ createdb datacube
 +
$ psql datacube -c 'create extension postgis'
 +
$ mkdir ~/.config/datacube 
 +
$ cat << EOF >  ~/.config/datacube/datacube.conf
 +
[datacube]
 +
db_database: datacube
 +
 
 +
# A blank host will use a local socket. Specify a hostname (such as localhost) to use TCP.
 +
db_hostname:
 +
 
 +
# Credentials are optional: you might have other Postgres authentication configured.
 +
# The default username otherwise is the current user id.
 +
db_username: user
 +
db_password: user
 +
EOF
 +
 
 +
                   
 +
user@osgeolive:~$ DATACUBE_CONFIG_PATH=~/.config/datacube/datacube.conf datacube -v system init
 +
2022-07-26 04:35:51,760 4663 datacube INFO Running datacube command: /usr/bin/datacube -v system init
 +
Initialising database...
 +
2022-07-26 04:35:52,213 4663 datacube.drivers.postgres._core INFO Ensuring user roles.
 +
2022-07-26 04:35:52,270 4663 datacube.drivers.postgres._core INFO Creating schema.
 +
2022-07-26 04:35:52,271 4663 datacube.drivers.postgres._core INFO Creating tables.
 +
2022-07-26 04:35:53,006 4663 datacube.drivers.postgres._core INFO Creating triggers.
 +
2022-07-26 04:35:53,015 4663 datacube.drivers.postgres._core INFO Creating added column.
 +
2022-07-26 04:35:53,055 4663 datacube.drivers.postgres._core INFO Adding role grants.
 +
2022-07-26 04:35:53,066 4663 datacube.index.postgres.index INFO Adding default metadata types.
 +
Created.
 +
Checking indexes/views.
 +
2022-07-26 04:35:53,376 4663 datacube.drivers.postgres._api INFO Checking dynamic views/indexes. (rebuild views=True, indexes=False)
 +
Done.
 +
 
 +
</pre>
 +
 
 +
https://github.com/opendatacube/datacube-dataset-config
 +
https://github.com/opendatacube/datacube-dataset-config/tree/main/products
 +
 
 +
https://mybinder.org/v2/gh/opendatacube/odc-stac/develop?urlpath=lab/workspaces/demo
 +
 
 +
https://notebooks.gesis.org/binder/jupyter/user/opendatacube-odc-stac-cnk43e3t/lab/workspaces/demo/tree/notebooks/stac-load-S2-ms.ipynb
 +
 
 +
https://datacube-core.readthedocs.io/en/latest/installation/indexing-data/step-guide.html
 +
 
 +
18June2022 - opendatacube notebooks have very current software under development. Finding the stable parts is a challenge. #osgeolive '''alpha1'''  python-geo environment is surprisingly current, and matches a lot of ODC team dev environments including Folium. BUT two important parts are different: odc-devs have more DASK components (multi-processing chains); they have extended rasterio parts that I have not seen here yet. Others not present on alpha1 are
 +
JupyterLab parts - we are not going to use those  (I read in the reviews late last year JupyterLab still has quality problems). ipyleaflet is developed with JupyterLab.. I am not confident ipyleaflet will work.. maybe. The alpha1 dot-deb packaging simplifies setup, MUCH simpler than their dev setup, and we have no DeepLearning ML GPU libs here.
 +
 
 +
Minimal datacube install from dot-deb 
 +
    $apt install  python3-xarray  python3-cfgrib  python3-datacube
 +
 
 +
    Packages that will be installed: python3-cachetools  python3-cloudpickle  python3-dask python3-datacube  python3-distributed  python3-fsspec python3-heapdict  python3-locket  python3-msgpack python3-partd  python3-psutil  python3-sortedcontainers python3-tblib  python3-toolz python3-tornado  python3-zict
 +
 
 +
=== Jupyter R Install ===
 +
<pre>
 +
dbb@x10jam:~$ sudo apt install r-cran-sf
 +
Reading package lists... Done
 +
Building dependency tree... Done
 +
Reading state information... Done
 +
 
 +
The following additional packages will be installed:
 +
  bzip2-doc gfortran gfortran-11 libblas-dev libbz2-dev libc-ares2 libgfortran-11-dev libjs-d3 libjs-es5-shim
 +
  libjs-highlight.js libjs-jquery-datatables libjs-jquery-datatables-extensions libjs-jquery-selectize.js libjs-json
 +
  libjs-microplugin.js libjs-modernizr libjs-prettify libjs-sifter.js libjs-twitter-bootstrap-datepicker liblapack-dev
 +
  libnlopt0 libnode72 libreadline-dev libudunits2-0 libudunits2-data littler node-highlight.js node-html5shiv
 +
  node-normalize.css nodejs nodejs-doc r-base-core r-base-dev r-cran-abind r-cran-acepack r-cran-askpass r-cran-assertthat
 +
  r-cran-backports r-cran-base64enc r-cran-bench r-cran-bh r-cran-bit r-cran-bit64 r-cran-bitops r-cran-blob r-cran-boot
 +
  r-cran-brio r-cran-broom r-cran-cachem r-cran-callr r-cran-caret r-cran-catools r-cran-cba r-cran-checkmate r-cran-chron
 +
  r-cran-class r-cran-classint r-cran-cli r-cran-cliapp r-cran-cluster r-cran-codetools r-cran-colorspace r-cran-commonmark
 +
  r-cran-conquer r-cran-covr r-cran-cpp11 r-cran-crayon r-cran-crosstalk r-cran-curl r-cran-cvst r-cran-data.table r-cran-dbi
 +
  r-cran-dbitest r-cran-dbplyr r-cran-ddalpha r-cran-decor r-cran-deldir r-cran-deoptimr r-cran-desc r-cran-diffobj
 +
  r-cran-digest r-cran-dimred r-cran-domc r-cran-dplyr r-cran-drr r-cran-dt r-cran-dygraphs r-cran-e1071 r-cran-earth
 +
  r-cran-ellipsis r-cran-evaluate r-cran-fansi r-cran-farver r-cran-fastica r-cran-fastmap r-cran-fastmatch r-cran-filehash
 +
  r-cran-fnn r-cran-fontbitstreamvera r-cran-fontliberation r-cran-fontquiver r-cran-foreach r-cran-foreign r-cran-formula
 +
  r-cran-fs r-cran-fts r-cran-furrr r-cran-future r-cran-future.apply r-cran-gdata r-cran-generics r-cran-geometry
 +
  r-cran-geosphere r-cran-ggplot2 r-cran-gh r-cran-gitcreds r-cran-globals r-cran-glue r-cran-goftest r-cran-gower
 +
  r-cran-gplots r-cran-gridextra r-cran-gtable r-cran-gtools r-cran-hexbin r-cran-highr r-cran-hmisc r-cran-hms
 +
  r-cran-htmltable r-cran-htmltools r-cran-htmlwidgets r-cran-httpuv r-cran-httr r-cran-igraph r-cran-ini r-cran-interp
 +
  r-cran-ipred r-cran-isoband r-cran-iterators r-cran-jpeg r-cran-jsonlite r-cran-kernlab r-cran-kernsmooth r-cran-knitr
 +
  r-cran-ks r-cran-labeling r-cran-later r-cran-lattice r-cran-latticeextra r-cran-lava r-cran-lazyeval r-cran-lifecycle
 +
  r-cran-linprog r-cran-listenv r-cran-littler r-cran-lme4 r-cran-lobstr r-cran-logcondens r-cran-lpsolve r-cran-lubridate
 +
  r-cran-lwgeom r-cran-magic r-cran-magrittr r-cran-mapproj r-cran-maps r-cran-maptools r-cran-markdown r-cran-mass
 +
  r-cran-matrix r-cran-matrixmodels r-cran-matrixstats r-cran-mclust r-cran-mda r-cran-memoise r-cran-mets r-cran-mgcv
 +
  r-cran-mime r-cran-minqa r-cran-misc3d r-cran-mlbench r-cran-mlmetrics r-cran-mockery r-cran-mockr r-cran-modeldata
 +
  r-cran-modelmetrics r-cran-multcomp r-cran-multicool r-cran-munsell r-cran-mvtnorm r-cran-nlme r-cran-nloptr r-cran-nnet
 +
  r-cran-numderiv r-cran-openssl r-cran-palmerpenguins r-cran-parallelly r-cran-pillar r-cran-pkgbuild r-cran-pkgconfig
 +
  r-cran-pkgkitten r-cran-pkgload r-cran-plogr r-cran-plot3d r-cran-plotmo r-cran-plotrix r-cran-pls r-cran-plyr r-cran-png
 +
  r-cran-polyclip r-cran-pracma r-cran-praise r-cran-prettycode r-cran-prettyunits r-cran-proc r-cran-processx r-cran-prodlim
 +
  r-cran-profmem r-cran-progress r-cran-progressr r-cran-promises r-cran-proxy r-cran-ps r-cran-purrr r-cran-quadprog
 +
  r-cran-quantmod r-cran-quantreg r-cran-r6 r-cran-ragg r-cran-rann r-cran-rappdirs r-cran-raster r-cran-rcolorbrewer
 +
  r-cran-rcpp r-cran-rcpparmadillo r-cran-rcppeigen r-cran-rcppprogress r-cran-rcpproll r-cran-recipes r-cran-rematch2
 +
  r-cran-reshape2 r-cran-rex r-cran-rgdal r-cran-rgl r-cran-rlang r-cran-rmarkdown r-cran-rmysql r-cran-robustbase r-cran-rocr
 +
  r-cran-rpart r-cran-rpostgresql r-cran-rprojroot r-cran-rsample r-cran-rspectra r-cran-rsqlite r-cran-rstudioapi r-cran-runit
 +
  r-cran-s2 r-cran-sandwich r-cran-sass r-cran-scales r-cran-selectr r-cran-setrng r-cran-sfsmisc r-cran-shiny r-cran-slider
 +
  r-cran-sodium r-cran-sourcetools r-cran-sp r-cran-sparsem r-cran-spatial r-cran-spatstat r-cran-spatstat.core
 +
  r-cran-spatstat.data r-cran-spatstat.geom r-cran-spatstat.linnet r-cran-spatstat.sparse r-cran-spatstat.utils r-cran-squarem
 +
  r-cran-statmod r-cran-stringi r-cran-stringr r-cran-survival r-cran-svglite r-cran-sys r-cran-systemfonts
 +
  r-cran-teachingdemos r-cran-tensor r-cran-terra r-cran-testit r-cran-testthat r-cran-textshaping r-cran-th.data r-cran-tibble
 +
  r-cran-tidyr r-cran-tidyselect r-cran-tikzdevice r-cran-timedate r-cran-timereg r-cran-timeseries r-cran-tinytex
 +
  r-cran-tseries r-cran-ttr r-cran-units r-cran-utf8 r-cran-vctrs r-cran-vdiffr r-cran-viridis r-cran-viridislite r-cran-waldo
 +
  r-cran-warp r-cran-webutils r-cran-withr r-cran-wk r-cran-wkutils r-cran-xfun r-cran-xml r-cran-xml2 r-cran-xtable r-cran-xts
 +
  r-cran-yaml r-cran-zoo r-doc-html r-recommended ttf-bitstream-vera
 +
 
 +
 
 +
Get:1 http://mirrors.wikimedia.org/ubuntu jammy/main amd64 bzip2-doc all 1.0.8-5build1 [500 kB]
 +
 
 +
...                   
 +
Get:325 http://mirrors.wikimedia.org/ubuntu jammy/universe amd64 r-recommended all 4.1.2-1ubuntu2 [2,624 B] 
 +
                 
 +
Fetched 299 MB in 12min 45s (391 kB/s)                                                                                         
 +
Extracting templates from packages: 100%
 +
Selecting previously unselected package bzip2-doc.
 +
(Reading database ... 438665 files and directories currently installed.)
 +
 
 +
Setting up libjs-json (0~20210531+~1.0.8-1) ...
 +
Setting up r-cran-bh (1.74.0-2) ...
 +
Setting up bzip2-doc (1.0.8-5build1) ...
 +
Setting up ttf-bitstream-vera (1.10-8.2) ...
 +
Setting up r-doc-html (4.1.2-1ubuntu2) ...
 +
Setting up libgfortran-11-dev:amd64 (11.2.0-19ubuntu1) ...
 +
Setting up libjs-sifter.js (0.6.0+dfsg-2) ...
 +
Setting up libjs-twitter-bootstrap-datepicker (1.3.1+dfsg1-4.1) ...
 +
Setting up node-html5shiv (3.7.3+dfsg-4) ...
 +
Setting up libnlopt0:amd64 (2.7.1-3build1) ...
 +
Setting up libjs-microplugin.js (0.0.3+dfsg-1.1) ...
 +
Setting up libc-ares2:amd64 (1.18.1-1build1) ...
 +
Setting up libreadline-dev:amd64 (8.1.2-1) ...
 +
Setting up libjs-modernizr (2.6.2+ds1-4) ...
 +
Setting up libnode72:amd64 (12.22.9~dfsg-1ubuntu3) ...
 +
Setting up libjs-jquery-datatables (1.11.4+dfsg-1) ...
 +
Setting up libjs-es5-shim (4.6.4-1) ...
 +
Setting up r-base-core (4.1.2-1ubuntu2) ...
 +
 
 +
Creating config file /etc/R/Renviron with new version
 +
 
 +
</pre>
 +
 
 +
Note:  https://isciences.gitlab.io/exactextractr/
 +
 
 +
=== pangeo test install ===
 +
 
 +
https://github.com/pangeo-data/pangeo-tutorial-gallery
 +
 
 +
<pre>
 +
$ pip install --user geocube
 +
Collecting geocube
 +
  Downloading geocube-0.3.2-py3-none-any.whl (20 kB)
 +
Requirement already satisfied: appdirs in /usr/lib/python3/dist-packages (from geocube) (1.4.4)
 +
Requirement already satisfied: pyproj>=2 in /usr/lib/python3/dist-packages (from geocube) (3.3.0)
 +
Requirement already satisfied: click>=6.0 in /usr/lib/python3/dist-packages (from geocube) (8.0.3)
 +
Collecting rioxarray>=0.4
 +
  Downloading rioxarray-0.11.1.tar.gz (49 kB)
 +
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.7/49.7 KB 283.0 kB/s eta 0:00:00
 +
  Installing build dependencies ... done
 +
  Getting requirements to build wheel ... done
 +
  Preparing metadata (pyproject.toml) ... done
 +
Collecting xarray>=0.17
 +
  Downloading xarray-2022.3.0-py3-none-any.whl (870 kB)
 +
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 870.9/870.9 KB 388.3 kB/s eta 0:00:00
 +
Requirement already satisfied: numpy>=1.20 in /usr/lib/python3/dist-packages (from geocube) (1.21.5)
 +
Requirement already satisfied: scipy in /usr/lib/python3/dist-packages (from geocube) (1.8.0)
 +
Requirement already satisfied: geopandas>=0.7 in /usr/lib/python3/dist-packages (from geocube) (0.10.2)
 +
Collecting odc-geo
 +
  Downloading odc_geo-0.2.1-py3-none-any.whl (102 kB)
 +
    ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.8/102.8 KB 279.6 kB/s eta 0:00:00
 +
Requirement already satisfied: rasterio in /usr/lib/python3/dist-packages (from geocube) (1.2.10)
 +
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from rioxarray>=0.4->geocube) (21.3)
 +
Requirement already satisfied: pandas>=1.1 in /usr/lib/python3/dist-packages (from xarray>=0.17->geocube) (1.3.5)
 +
Requirement already satisfied: affine in /usr/lib/python3/dist-packages (from odc-geo->geocube) (2.3.0)
 +
Requirement already satisfied: cachetools in /usr/lib/python3/dist-packages (from odc-geo->geocube) (5.0.0)
 +
Requirement already satisfied: shapely in /usr/lib/python3/dist-packages (from odc-geo->geocube) (1.8.0)
 +
Building wheels for collected packages: rioxarray
 +
  Building wheel for rioxarray (pyproject.toml) ... done
 +
  Created wheel for rioxarray: filename=rioxarray-0.11.1-py2.py3-none-any.whl size=50942 sha256=61bb0bfd8815420dc5129f7c7a14afff11796b6978c976a5b3001e54b16ac5e4
 +
  Stored in directory: /home/dbb/.cache/pip/wheels/65/e9/a4/20811fc3ecaddf11c40d1e649d0d1bbe3cbbbd71559ca3cd9b
 +
Successfully built rioxarray
 +
Installing collected packages: xarray, odc-geo, rioxarray, geocube
 +
  WARNING: The script geocube is installed in '/home/dbb/.local/bin' which is not on PATH.
 +
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
 +
Successfully installed geocube-0.3.2 odc-geo-0.2.1 rioxarray-0.11.1 xarray-2022.3.0
 +
 
 +
</pre>
 +
 
 +
=== env ===
 +
 
 +
    TODO pytest tests integration_tests in datacube-core
 +
 
 +
 
 +
<pre>
 +
 
 +
GDAL            3.4.3
 +
numpy          1.21.5
 +
scipy          1.8.0
 +
dask            2022.1.0+dfsg
 +
pandas          1.3.5
 +
geopandas      0.10.2
 +
ipython        7.31.1
 +
matplotlib      3.5.1
 +
pyproj          3.3.0
 +
scikit-learn 0.23.2
 +
seaborn        0.11.2
 +
datacube        0.0.0
 +
 
 +
APScheduler 0.0.0
 +
Automat 20.2.0
 +
Babel 2.8.0
 +
Brotli 1.0.9
 +
Cartopy 0.0.0
 +
Cheetah3 3.2.6
 +
Deprecated 1.2.13
 +
Django 2.2.24
 +
EOxServer 1.1.2
 +
Faker 0.9.3
 +
Fiona 1.8.21
 +
Flask 2.0.1
 +
GeoAlchemy2 0.10.2
 +
GeoNode 3.3.2.post2
 +
HeapDict 1.0.1
 +
Jinja2 3.0.3
 +
Mako 1.1.3
 +
MapProxy 1.15.1
 +
Markdown 3.3.6
 +
MarkupSafe 2.0.1
 +
Nik4 1.7.0
 +
OWSLib 0.25.0
 +
Pillow 9.0.1
 +
PyGObject 3.42.1
 +
PyHamcrest 2.0.2
 +
PyJWT 2.3.0
 +
PyNaCl 1.5.0
 +
PyOpenGL 3.1.5
 +
PyQt5-sip 12.9.1
 +
PyQt5 5.15.6
 +
PyYAML 5.4.1
 +
Pygments 2.11.2
 +
QScintilla 2.11.6
 +
Rtree 0.9.7
 +
SQLAlchemy 1.4.31
 +
SecretStorage 3.3.1
 +
Send2Trash 1.8.1b0
 +
Shapely 1.8.0
 +
Twisted 22.1.0
 +
Unidecode 1.3.3
 +
Werkzeug 2.0.2
 +
adal 1.2.2
 +
affine 2.3.0
 +
aiohttp 3.8.1
 +
aioredis 1.3.1
 +
aiosignal 1.2.0
 +
amqp 5.0.9
 +
anyjson 0.3.3
 +
appdirs 1.4.4
 +
apt-xapian-index 0.49
 +
argon2-cffi 21.1.0
 +
async-timeout 4.0.1
 +
attrs 21.2.0
 +
avro 1.11.0
 +
azure-template 0.0.18b3
 +
backcall 0.2.0
 +
bcrypt 3.2.0
 +
beautifulsoup4 4.10.0
 +
billiard 3.6.4.0
 +
bleach 4.1.0
 +
blinker 1.4
 +
boto3 1.20.34
 +
botocore 1.23.34
 +
branca 0.4.2
 +
cachetools 5.0.0
 +
celery 5.2.3
 +
certifi 2020.6.20
 +
cftime 1.5.2
 +
chardet 4.0.0
 +
charset-normalizer 2.0.6
 +
click-didyoumean 0.0.3
 +
click-plugins 1.1.1
 +
click-repl 0.2.0
 +
click 8.0.3
 +
cligj 0.7.2
 +
cloudpickle 2.0.0
 +
colorama 0.4.4
 +
command-not-found 0.3
 +
constantly 15.1.0
 +
coverage 6.2
 +
croniter 1.0.15
 +
cryptography 3.4.8
 +
cupshelpers 1.0
 +
cycler 0.11.0
 +
dbus-python 1.2.18
 +
decorator 4.4.2
 +
defer 1.0.6
 +
defusedxml 0.7.1
 +
deprecation 2.0.7
 +
descartes 1.1.0
 +
distributed 2022.1.0+ds.1
 +
distro-info =1.1build1
 +
distro 1.7.0
 +
dj-database-url 0.5.0
 +
dj-pagination 2.5.0
 +
dnspython 2.1.0
 +
docker 5.0.3
 +
drf-extensions 0.7.1
 +
drf-spectacular 0.21.1
 +
drf-writable-nested 0.6.3
 +
dynamic-rest 2.0.0
 +
elasticsearch 7.16.2
 +
entrypoints 0.4
 +
ephem 4.1.3
 +
factory-boy 2.11.1
 +
flake8 4.0.1
 +
flaky 3.7.0
 +
folium 0.0.0
 +
fonttools 4.29.1
 +
frozenlist 1.2.0
 +
fs 2.4.12
 +
fsspec 2022.1.0
 +
future 0.18.2
 +
galternatives 1.0.8
 +
geographiclib 1.52
 +
geoip2 2.9.0
 +
geojson 2.5.0
 +
geolinks 0.2.0
 +
geonode-announcements 2.0.2
 +
geonode-avatar 5.0.7
 +
geonode-dialogos 3.0.1
 +
geonode-django-activity-stream 0.10.0
 +
geonode-oauth-toolkit 2.2.2
 +
geonode-user-messages 2.0.2
 +
geoserver-restconfig 2.0.7
 +
gisdata 0.5.4
 +
glob2 0.5
 +
gn-arcrest 10.5.4
 +
gn-gsimporter 2.0.1
 +
gps 3.22
 +
greenlet 1.1.2
 +
gunicorn 20.1.0
 +
hiredis 1.0.1
 +
html5lib 1.1
 +
httplib2 0.20.2
 +
humanize 0.0.0
 +
hyperlink 21.0.0
 +
idna 3.3
 +
importlib-metadata 4.6.4
 +
incremental 21.3.0
 +
inflection 0.3.1
 +
iniconfig 1.1.1
 +
invoke 1.4.1
 +
iotop 0.6
 +
ipykernel 6.7.0
 +
ipyleaflet 0.17.0
 +
ipython_genutils 0.2.0
 +
ipywidgets 6.0.0
 +
iso8601 1.0.2
 +
isodate 0.6.1
 +
itsdangerous 2.1.0
 +
jdcal 1.0
 +
jedi 0.18.0
 +
jeepney 0.7.1
 +
jmespath 0.10.0
 +
joblib 0.17.0
 +
jsonfield 3.1.0
 +
jsonref 0.1
 +
jsonschema 3.2.0
 +
jupyter-client 7.1.2
 +
jupyter-core 4.9.1
 +
jupyter-packaging 0.11.1
 +
jupyterlab-pygments 0.1.2
 +
jwcrypto 1.0
 +
keyring 23.5.0
 +
kiwisolver 1.3.2
 +
kombu 5.2.3
 +
language-selector 0.1
 +
lark-parser 0.12.0
 +
launchpadlib 1.10.16
 +
lazr.restfulclient 0.14.4
 +
lazr.uri 1.0.6
 +
locket 0.2.1
 +
lubuntu-update-notifier 0.4.dev0
 +
lxml 4.8.0
 +
lz4 3.1.3+dfsg
 +
mapnik 3.0.23
 +
mappyfile 0.9.7
 +
mapscript 7.6.4
 +
matplotlib-inline 0.1.3
 +
maxminddb 2.0.3
 +
mccabe 0.6.1
 +
mercantile 1.1.5
 +
mock 4.0.3
 +
more-itertools 8.10.0
 +
mpmath 0.0.0
 +
msal-extensions 1.0.0
 +
msal 1.17.0
 +
msgpack 1.0.3
 +
msrest 0.6.21
 +
msrestazure 0.6.4
 +
multidict 5.1.0
 +
munch 2.5.0
 +
nbclient 0.5.6
 +
nbconvert 6.4.0
 +
nbformat 5.1.3
 +
nest-asyncio 1.5.4
 +
netCDF4 1.5.8
 +
netifaces 0.11.0
 +
networkx 2.4
 +
notebook 6.4.8
 +
oauthlib 3.2.0
 +
osmapi 3.0.0
 +
osmium 3.2.0
 +
packaging 21.3
 +
palettable 3.3.0
 +
pandocfilters 1.5.0
 +
paramiko 2.9.3
 +
parse-type 0.5.6
 +
parse 1.19.0
 +
parso 0.8.1
 +
partd 1.2.0
 +
pbr 5.8.0
 +
pexpect 4.8.0
 +
pickleshare 0.7.5
 +
pilkit 2.0
 +
pinax-notifications 6.0.0
 +
pinax-ratings 4.0.0
 +
plotly 5.4.0
 +
pluggy 0.13.0
 +
ply 3.11
 +
portalocker 2.2.1
 +
prometheus-client 0.9.0
 +
prompt-toolkit 3.0.28
 +
psutil 5.9.0
 +
psycopg2 2.8.6
 +
ptyprocess 0.7.0
 +
py 1.10.0
 +
pyOpenSSL 21.0.0
 +
pyasn1-modules 0.2.1
 +
pyasn1 0.4.8
 +
pycairo 1.20.1
 +
pycodestyle 2.8.0
 +
pycountry 20.7.3
 +
pycql 0.0.8
 +
pycsw 2.6.1
 +
pycups 2.0.1
 +
pydantic 1.8.2
 +
pyflakes 2.4.0
 +
pygeoapi 0.12.0
 +
pylibmc 1.5.2
 +
pyows 0.2.6
 +
pyparsing 2.4.7
 +
pyrsistent 0.18.1
 +
pysal 2.1.0
 +
pyserial 3.5
 +
pyshp 2.2.0
 +
pytest-bdd 3.2.1
 +
pytest-django 3.5.1
 +
pytest 6.2.5
 +
python-apt 2.3.0+ubuntu2
 +
python-crontab 2.6.0
 +
python-dateutil 2.8.1
 +
python-debian 0.1.43ubuntu1
 +
python-etcd 0.4.5
 +
python-memcached 1.58
 +
python-mimeparse 1.6.0
 +
python-slugify 4.0.0
 +
python3-openid 3.2.0
 +
pytz 2022.1
 +
pywps 4.5.1
 +
pyxdg 0.27
 +
pyzmq 22.3.0
 +
rasterio 1.3.0
 +
redis 3.5.3
 +
requests-oauthlib 1.3.0
 +
requests-toolbelt 0.9.1
 +
requests 2.25.1
 +
s3transfer 0.5.0
 +
schema 0.6.7
 +
selenium 4.0.0a1
 +
service-identity 18.1.0
 +
sherlock 0.3.2
 +
six 1.16.0
 +
smart-open 5.2.1
 +
snuggs 1.4.7
 +
sortedcontainers 2.1.0
 +
soupsieve 2.3.1
 +
sqlparse 0.4.2
 +
sympy 1.9
 +
systemd-python 234
 +
tblib 1.7.0
 +
tenacity 6.3.1
 +
terminado 0.13.1
 +
testpath 0.5.0
 +
threadpoolctl 3.1.0
 +
timeout-decorator 0.5.0
 +
tinydb 3.15.2
 +
toml 0.10.2
 +
tomlkit 0.9.2
 +
toolz 0.11.2
 +
tornado 6.1
 +
tqdm 4.57.0
 +
traitlets 5.1.1
 +
traittypes 0.2.1
 +
typing-extensions 3.10.0.2
 +
tzlocal 2.1
 +
ua-parser 0.10.0
 +
uamqp 1.5.1
 +
ubuntu-advantage-tools 27.7
 +
ubuntu-drivers-common 0.0.0
 +
ufoLib2 0.13.1
 +
ufw 0.36.1
 +
unattended-upgrades 0.1
 +
unicodecsv 0.14.1
 +
unicodedata2 14.0.0
 +
uritemplate 3.0.1
 +
urllib3 1.26.5
 +
usb-creator 0.3.7
 +
user-agents 2.2.0
 +
vine 5.0.0
 +
wadllib 1.3.6
 +
wcwidth 0.2.5
 +
webencodings 0.5.1
 +
websocket-client 1.2.3
 +
websockets 9.1
 +
widgetsnbextension 2.0.0
 +
wrapt 1.13.3
 +
wxPython 4.0.7
 +
xarray 0.16.1
 +
xdg 5
 +
xkit 0.0.0
 +
xmljson 0.1.9
 +
xmltodict 0.12.0
 +
xyzservices 2022.6.0
 +
yarl 1.7.2
 +
zict 2.0.0
 +
zipp 1.0.0
 +
zipstream 1.1.4
 +
zope.interface 5.4.0
 +
 
 +
django-allauth 0.47.0
 +
django-appconf 1.0.3
 +
django-autocomplete-light 3.5.1
 +
django-bootstrap3-datetimepicker-2 2.8.2
 +
django-celery-beat 2.2.1
 +
django-celery-results 2.0.0
 +
django-cors-headers 3.11.0
 +
django-downloadview 2.3.0
 +
django-filter 2.4.0
 +
django-floppyforms 1.9.0
 +
django-forms-bootstrap 3.1.0
 +
django-geonode-mapstore-client 3.3.2
 +
django-grappelli 2.15.1
 +
django-guardian 2.4.0
 +
django-haystack 3.1.1
 +
django-imagekit 4.1.0
 +
django-invitations 1.9.3
 +
django-ipware 4.0.0
 +
django-js-asset 1.2.2
 +
django-jsonfield-backport 1.0.2
 +
django-leaflet 0.27.1
 +
django-markdownify 0.9.1
 +
django-model-utils 0.0.0
 +
django-modeltranslation 0.17.5
 +
django-mptt 0.13.2
 +
django-polymorphic 3.1.0
 +
django-recaptcha 2.0.6
 +
django-storages 1.12.3
 +
django-taggit 2.1.0
 +
django-tastypie 0.14.4
 +
django-timezone-field 5.0
 +
django-tinymce 3.3.0
 +
django-treebeard 4.5.1
 +
django-user-agents 0.4.0
 +
django-uuid-upload-path 1.0.0
 +
django-widget-tweaks 1.4.8
 +
djangorestframework-gis 0.18
 +
djangorestframework-guardian 0.3.0
 +
djangorestframework 3.12.4
  
 +
azure-agrifood-farming 1.0.0b1
 +
azure-ai-anomalydetector 3.0.0b3
 +
azure-ai-formrecognizer 3.2.0b2
 +
azure-ai-language-conversations 1.0.0b1
 +
azure-ai-language-questionanswering 1.0.0
 +
azure-ai-metricsadvisor 1.0.1
 +
azure-ai-textanalytics 5.2.0b3
 +
azure-ai-translation-document 1.0.0b6
 +
azure-appconfiguration 1.3.0
 +
azure-applicationinsights 0.1.0
 +
azure-batch 11.0.0
 +
azure-cognitiveservices-anomalydetector 0.3.0
 +
azure-cognitiveservices-formrecognizer 0.1.1
 +
azure-cognitiveservices-knowledge-qnamaker 0.3.0
 +
azure-cognitiveservices-language-luis 0.7.0
 +
azure-cognitiveservices-language-spellcheck 2.0.0
 +
azure-cognitiveservices-language-textanalytics 0.2.0
 +
azure-cognitiveservices-personalizer 0.1.0
 +
azure-cognitiveservices-search-autosuggest 0.2.0
 +
azure-cognitiveservices-search-customimagesearch 0.2.0
 +
azure-cognitiveservices-search-customsearch 0.3.0
 +
azure-cognitiveservices-search-entitysearch 2.0.0
 +
azure-cognitiveservices-search-imagesearch 2.0.0
 +
azure-cognitiveservices-search-newssearch 2.0.0
 +
azure-cognitiveservices-search-videosearch 2.0.0
 +
azure-cognitiveservices-search-visualsearch 0.2.0
 +
azure-cognitiveservices-search-websearch 2.0.0
 +
azure-cognitiveservices-vision-computervision 0.9.0
 +
azure-cognitiveservices-vision-contentmoderator 1.0.0
 +
azure-cognitiveservices-vision-customvision 3.1.0
 +
azure-cognitiveservices-vision-face 0.5.0
 +
azure-common 1.1.28
 +
azure-communication-chat 1.1.0
 +
azure-communication-identity 1.0.1
 +
azure-communication-networktraversal 1.0.0b1
 +
azure-communication-phonenumbers 1.0.1
 +
azure-communication-sms 1.0.1
 +
azure-confidentialledger 1.0.0b1
 +
azure-containerregistry 1.0.0b7
 +
azure-core 1.20.1
 +
azure-data-tables 12.1.1
 +
azure-digitaltwins-core 1.1.0
 +
azure-eventgrid 4.7.0
 +
azure-eventhub-checkpointstoreblob-aio 1.1.4
 +
azure-eventhub-checkpointstoreblob 1.1.4
 +
azure-eventhub-checkpointstoretable 1.0.0b1
 +
azure-eventhub 5.6.2
 +
azure-graphrbac 0.61.1
 +
azure-identity 1.7.1
 +
azure-iot-deviceupdate 1.0.0b1
 +
azure-iot-modelsrepository 1.0.0b2
 +
azure-keyvault-administration 4.1.0b2
 +
azure-keyvault-certificates 4.4.0b2
 +
azure-keyvault-keys 4.5.0b5
 +
azure-keyvault-secrets 4.4.0b2
 +
azure-keyvault 4.1.0
 +
azure-loganalytics 0.1.0
 +
azure-media-analytics-edge 1.0.0b1
 +
azure-media-videoanalyzer-edge 1.0.0b3
 +
azure-messaging-webpubsubservice 1.0.0b3
 +
azure-mgmt-advisor 9.0.0
 +
azure-mgmt-agfood 1.0.0b1
 +
azure-mgmt-agrifood 1.0.0b1
 +
azure-mgmt-alertsmanagement 1.0.0
 +
azure-mgmt-apimanagement 2.1.0
 +
azure-mgmt-appconfiguration 2.0.0
 +
azure-mgmt-applicationinsights 1.0.0
 +
azure-mgmt-appplatform 6.1.0
 +
azure-mgmt-attestation 1.0.0
 +
azure-mgmt-authorization 2.0.0
 +
azure-mgmt-automanage 1.0.0b2
 +
azure-mgmt-automation 1.1.0b1
 +
azure-mgmt-avs 7.0.0b1
 +
azure-mgmt-azureadb2c 1.0.0b1
 +
azure-mgmt-azurearcdata 1.0.0
 +
azure-mgmt-azurestack 1.0.0
 +
azure-mgmt-azurestackhci 6.1.0b1
 +
azure-mgmt-baremetalinfrastructure 1.0.0
 +
azure-mgmt-batch 16.0.0
 +
azure-mgmt-batchai 7.0.0b1
 +
azure-mgmt-billing 6.0.0
 +
azure-mgmt-botservice 1.0.0
 +
azure-mgmt-cdn 11.0.0
 +
azure-mgmt-changeanalysis 1.0.0
 +
azure-mgmt-chaos 1.0.0b2
 +
azure-mgmt-cognitiveservices 12.0.0
 +
azure-mgmt-commerce 6.0.0
 +
azure-mgmt-communication 1.0.0
 +
azure-mgmt-compute 23.1.0
 +
azure-mgmt-confidentialledger 1.0.0b1
 +
azure-mgmt-confluent 2.0.0b1
 +
azure-mgmt-consumption 8.0.0
 +
azure-mgmt-containerinstance 9.1.0
 +
azure-mgmt-containerregistry 8.2.0
 +
azure-mgmt-containerservice 16.3.0
 +
azure-mgmt-core 1.3.0
 +
azure-mgmt-cosmosdb 7.0.0b2
 +
azure-mgmt-costmanagement 3.0.0
 +
azure-mgmt-customproviders 1.0.0
 +
azure-mgmt-databox 1.0.0
 +
azure-mgmt-databoxedge 1.0.0
 +
azure-mgmt-databricks 1.1.0b1
 +
azure-mgmt-datadog 2.0.0
 +
azure-mgmt-datafactory 2.0.0
 +
azure-mgmt-datalake-analytics 0.6.0
 +
azure-mgmt-datalake-store 1.0.0
 +
azure-mgmt-datamigration 10.0.0
 +
azure-mgmt-dataprotection 1.0.0b1
 +
azure-mgmt-datashare 1.0.0
 +
azure-mgmt-deploymentmanager 1.0.0
 +
azure-mgmt-deviceupdate 1.0.0b3
 +
azure-mgmt-devspaces 1.0.0b1
 +
azure-mgmt-devtestlabs 9.0.0
 +
azure-mgmt-digitaltwins 6.0.0
 +
azure-mgmt-dns 8.0.0
 +
azure-mgmt-documentdb 0.1.3
 +
azure-mgmt-edgegateway 0.1.0
 +
azure-mgmt-edgeorder 1.0.0b1
 +
azure-mgmt-elastic 1.0.0
 +
azure-mgmt-eventgrid 9.0.0
 +
azure-mgmt-eventhub 10.0.0
 +
azure-mgmt-extendedlocation 1.0.0
 +
azure-mgmt-fluidrelay 1.0.0b1
 +
azure-mgmt-frontdoor 1.0.0
 +
azure-mgmt-guestconfig 1.0.0b1
 +
azure-mgmt-hanaonazure 1.0.0
 +
azure-mgmt-hdinsight 9.0.0
 +
azure-mgmt-healthbot 1.0.0b1
 +
azure-mgmt-healthcareapis 1.1.0b1
 +
azure-mgmt-hybridcompute 7.0.0
 +
azure-mgmt-hybridkubernetes 1.1.0
 +
azure-mgmt-hybridnetwork 1.0.0
 +
azure-mgmt-imagebuilder 1.0.0b1
 +
azure-mgmt-iotcentral 9.0.0b1
 +
azure-mgmt-iothub 2.1.0
 +
azure-mgmt-iothubprovisioningservices 1.0.0
 +
azure-mgmt-keyvault 9.2.0
 +
azure-mgmt-kubernetesconfiguration 1.0.0b1
 +
azure-mgmt-kusto 2.1.0
 +
azure-mgmt-labservices 1.0.0
 +
azure-mgmt-loganalytics 11.0.0
 +
azure-mgmt-logic 9.0.0
 +
azure-mgmt-logz 1.0.0
 +
azure-mgmt-machinelearningcompute 1.0.0b1
 +
azure-mgmt-machinelearningservices 1.0.0
 +
azure-mgmt-maintenance 2.0.0
 +
azure-mgmt-managedservices 6.0.0
 +
azure-mgmt-managementgroups 1.0.0
 +
azure-mgmt-managementpartner 1.0.0
 +
azure-mgmt-maps 2.0.0
 +
azure-mgmt-marketplaceordering 1.1.0
 +
azure-mgmt-media 8.0.0
 +
azure-mgmt-mixedreality 1.0.0
 +
azure-mgmt-monitor 2.0.0
 +
azure-mgmt-msi 6.0.0b1
 +
azure-mgmt-netapp 5.1.0
 +
azure-mgmt-network 19.2.0
 +
azure-mgmt-notificationhubs 7.0.0
 +
azure-mgmt-operationsmanagement 1.0.0
 +
azure-mgmt-peering 1.0.0
 +
azure-mgmt-policyinsights 1.1.0b1
 +
azure-mgmt-portal 1.0.0
 +
azure-mgmt-powerbidedicated 1.0.0
 +
azure-mgmt-powerbiembedded 2.0.0
 +
azure-mgmt-privatedns 1.0.0
 +
azure-mgmt-purview 1.0.0
 +
azure-mgmt-quantum 1.0.0b2
 +
azure-mgmt-quota 1.0.0b2
 +
azure-mgmt-rdbms 10.0.0
 +
azure-mgmt-recoveryservices 2.0.0
 +
azure-mgmt-recoveryservicesbackup 3.0.0
 +
azure-mgmt-recoveryservicessiterecovery 1.0.0b1
 +
azure-mgmt-redhatopenshift 1.0.0
 +
azure-mgmt-redis 13.0.0
 +
azure-mgmt-redisenterprise 1.0.0
 +
azure-mgmt-regionmove 1.0.0b1
 +
azure-mgmt-relay 1.1.0
 +
azure-mgmt-reservations 1.0.0
 +
azure-mgmt-resource 20.0.0
 +
azure-mgmt-resourcegraph 8.1.0b1
 +
azure-mgmt-resourcehealth 1.0.0b1
 +
azure-mgmt-resourcemover 1.1.0b2
 +
azure-mgmt-scheduler 7.0.0b1
 +
azure-mgmt-search 8.0.0
 +
azure-mgmt-security 2.0.0b1
 +
azure-mgmt-securityinsight 1.0.0b1
 +
azure-mgmt-serialconsole 1.0.0
 +
azure-mgmt-servermanager 2.0.0
 +
azure-mgmt-servicebus 7.1.0
 +
azure-mgmt-servicefabric 2.0.0
 +
azure-mgmt-servicefabricmanagedclusters 1.0.0
 +
azure-mgmt-servicelinker 1.0.0b1
 +
azure-mgmt-signalr 1.0.0
 +
azure-mgmt-sql 3.0.1
 +
azure-mgmt-sqlvirtualmachine 1.0.0b1
 +
azure-mgmt-storage 19.0.0
 +
azure-mgmt-storagecache 1.1.0
 +
azure-mgmt-storageimportexport 1.0.0b1
 +
azure-mgmt-storagepool 1.0.0
 +
azure-mgmt-storagesync 1.0.0
 +
azure-mgmt-streamanalytics 1.0.0rc1
 +
azure-mgmt-subscription 1.0.0
 +
azure-mgmt-support 6.0.0
 +
azure-mgmt-synapse 2.1.0b2
 +
azure-mgmt-testbase 1.0.0b1
 +
azure-mgmt-timeseriesinsights 1.0.0
 +
azure-mgmt-trafficmanager 1.0.0b1
 +
azure-mgmt-videoanalyzer 1.0.0b3
 +
azure-mgmt-vmwarecloudsimple 1.0.0b1
 +
azure-mgmt-web 5.0.0
 +
azure-mgmt-webpubsub 1.0.0
 +
azure-mgmt-workloadmonitor 1.0.0b2
 +
azure-mixedreality-authentication 1.0.0b1
 +
azure-monitor-opentelemetry-exporter 1.0.0b6
 +
azure-monitor-query 1.0.1
 +
azure-purview-administration 1.0.0b2
 +
azure-purview-catalog 1.0.0b3
 +
azure-purview-scanning 1.0.0b3
 +
azure-schemaregistry-avroserializer 1.0.0b4
 +
azure-schemaregistry 1.0.0
 +
azure-search-documents 11.3.0b5
 +
azure-security-attestation 1.0.1
 +
azure-servicebus 7.4.0
 +
azure-servicefabric 8.0.0.0
 +
azure-servicemanagement-legacy 0.20.7
 +
azure-storage-blob-changefeed 12.0.0b3
 +
azure-storage-blob 12.9.0
 +
azure-storage-file-datalake 12.5.0
 +
azure-storage-file-share 12.6.0
 +
azure-storage-queue 12.1.6
 +
azure-synapse-accesscontrol 0.8.0
 +
azure-synapse-artifacts 0.10.0
 +
azure-synapse-managedprivateendpoints 0.5.0
 +
azure-synapse-monitoring 0.3.0
 +
azure-synapse-spark 0.8.0
 +
azure-synapse 0.1.1
 +
----------------------------- ----------------------------
 +
Package                      Version
 
</pre>
 
</pre>
  
−
=== Python Libraries Strategy Notes ===
+
<pre>
 +
http://ci.terria.io/main/
 +
https://datacube-core.readthedocs.io/en/latest/installation/database/setup.html
 +
https://docs.xarray.dev/en/stable/user-guide/plotting.html#robust
 +
https://github.com/opendatacube/datacube-core/releases/tag/1.8.7
 +
https://github.com/opendatacube/odc-geo/issues
 +
https://github.com/opendatacube/odc-geo/releases/tag/v0.2.0
 +
https://github.com/opendatacube/odc-stac/releases/tag/v0.3.0
 +
https://github.com/opendatacube/odc-stac/wiki/Generating-Rotated-Images-to-Save-Space
 +
https://github.com/opendatacube/odc-tools/blob/develop/apps/dc_tools/odc/apps/dc_tools/fs_to_dc.py
 +
https://maps.digitalearth.africa/
 +
https://odc-geo.readthedocs.io/en/latest/_api/odc.geo.xr.ODCExtensionDs.html
 +
https://odc-geo.readthedocs.io/en/latest/_api/odc.geo.xr.ODCExtension.html
 +
https://odc-geo.readthedocs.io/en/latest/raster-tools.html#plotting-on-a-map
 +
https://odc-stac-demo.netlify.app/
 +
https://sandbox.digitalearth.africa/hub/login?next=%2Fhub%2F
 +
https://www.usgs.gov/landsat-missions/landsat-collection-2-level-2-science-products
  
−
''Some priority python for the Jupyter Notebooks of 2022''
+
Alternately, you can configure the ODC connection to Postgres using environment variables:
 +
DB_HOSTNAME
 +
DB_USERNAME
 +
DB_PASSWORD
 +
DB_DATABASE
  
−
* '''GeoPandas''' (and pySAL) : direct link to '''GEOS 3.10''', popular extension to Pandas data science; please recall that pySAL is merging into the GeoPandas project overall;  GeoPandas+pySAL use matplotlib graphics base, prefers Folium for web maps (?)
+
</pre>
  
−
* '''Numpy''' & '''Pandas''' : fundamental base packaging, versions matter
 
  
−
* PostGIS writing done with '''pyscopg2''' (low-level) or '''geoalchemy2''' (ORM based on sqlalchemy)
+
DevInstall via Paul Haesler on Slack #open-data-cube
  
−
* '''Scikit-Learn''' :  original machine-learning library by international research team led by French math; stands the tests of time; integrated with SciPy for Fortran, BLAS, LAPACK performant libs; standard numpy for data allocation; important primitives like PCA, SVM, and KNN are optimized in practical ways, and then benchmarked against pure Fortran or C++ for comparison
+
<pre>
 +
# install numpy, pyproj and rasterio from Ubuntu distro via apt
 +
sudo apt-get install python3-numpy python3-rasterio python3-pyproj
  
−
* '''boto''' for AWS S3, other data store transfer; can use AWS credentials (or Hetzner, or..) modern version using pip installed locally
+
# Create virtualenv inheriting numpy and rasterio from system site-packages.
 +
cd ~/venvs
 +
virtualenv -p `which python3` --system-site-packages odc
  
−
* '''rasterio''' :  modern geoTIFF handling; sophisticated library design; not easy to build from source
+
# Activate new virtualenv
 +
source odc/bin/activate
  
−
* '''Shapely''' & '''Fiona''' : direct link to libGEOS/ogr from python; example Notebooks in place 
+
# Compile and install cf-units from source:
 +
cd ~/src
 +
git clone git@github.com:SciTools/cf-units.git cf-units
 +
cd cf-units
 +
pip install -e .
  
−
* ''mapclassify'' : part of pySAL, versions must match with geopandas setup
+
# install datacube-core from source (from pypi should be fine after next release)
 +
cd ~/src/datacube/core
 +
pip install -e .[all]
 +
</pre>
  
−
*  ''lightgbm'' : MSFT python, tricky to install, might have to wait until other versions are sorted here
+
=== odc-geo ===
 +
<pre>
 +
$ pip install --user --upgrade-strategy only-if-needed odc-geo
 +
Collecting odc-geo
 +
  Downloading odc_geo-0.1.1-py3-none-any.whl (81 kB)
 +
    |                              | 81 kB 254 kB/s
 +
Requirement already satisfied: affine in ./.local/lib/python3.8/site-packages (from odc-geo) (2.3.0)
 +
Collecting cachetools
 +
  Downloading cachetools-5.0.0-py3-none-any.whl (9.1 kB)
 +
Requirement already satisfied: numpy in ./.local/lib/python3.8/site-packages (from odc-geo) (1.21.5)
 +
Requirement already satisfied: pyproj in /usr/lib/python3/dist-packages (from odc-geo) (3.2.1)
 +
Requirement already satisfied: shapely in /usr/lib/python3/dist-packages (from odc-geo) (1.8.0)
 +
Installing collected packages: cachetools, odc-geo
 +
Successfully installed cachetools-5.0.0 odc-geo-0.1.1
  
−
* libosmium2-dev, libosmium2-doc, python3-pyosmium, osmium-tool : a C++ framework for working with OSM data files; osmium library bindings for Python 3
+
</pre>
  
−
* libmapnik3.1, python3-mapnik : mapnik rendering engine for OSM maps and others
+
=== odc-stac ===
  
−
* LiDAR "no" : this is slightly painful since modern commercial GIS is flooded with this tech, delivered by advanced Western companies seeking something novel to sell. Little to no "libre" support and that includes here.
+
<pre>
 +
## from STAC-load-ms-S2.ipynb
 +
import dask.distributed
 +
import dask.utils
 +
import numpy as np
 +
import planetary_computer as pc
 +
import xarray as xr
 +
from IPython.display import display
 +
from pystac_client import Client
 +
from odc.stac import configure_rio, stac_load
  
−
* DeepLearning "no" :  deep learning has taken over in the modern AI/ML research, for funding and hype, but not on this disk. Please note that the ML libraries on osgeolive are completely useful, accurate and performant today.. no need for a constant and fast Internet connection, nor a compute infrastructure the size of a large naval ship, or special impenetrable models that can only be built by those machines. [http://blog.light42.com/wordpress/?p=3564 -LINK-]  Serious and in-demand applications like "facial recognition" are not included here. Many real applications to remote sensing are built using DeepLearning, but those are not available on this disk. Some benefits of not including DeepLearning : no dependencies on tricky GPU libs; not suffering from fast-updates of important ML core libraries; avoid black-box training models, and their disk space; avoid software patent problems in Western Nations.
+
## from STAC-load-e84-aws.ipynb
 +
import dask.distributed
 +
import folium
 +
import folium.plugins
 +
import geopandas as gpd
 +
import shapely.geometry
 +
from IPython.display import HTML, display
 +
from pystac_client import Client
 +
from odc.stac import configure_rio, stac_load
  
−
"''I feel that a common and well-known distro like this is an opportunity to build engineering cooperation, between teams that may not interact otherwise.. like the physical code sprints''"  -darkblueb in California
+
</pre>
  
−
Example Notebooks in previous OSGeoLive  [https://git.osgeo.org/gitea/darkblueb/ol14_ipynb_misc -LINK-]
+
== Focal Notes ==
  
−
--
 
  
 
=== Focal Setup Steps ===
 
=== Focal Setup Steps ===
Line 129: Line 1,255:
 
<pre>
 
<pre>
 
   ## note gdal & postgis are installed at an earlier step via package dot-deb
 
   ## note gdal & postgis are installed at an earlier step via package dot-deb
 +
  ## -------------------------------------------------------------------------
 
   591  sudo apt install python3-dev python3-pip
 
   591  sudo apt install python3-dev python3-pip
 
   405  sudo apt install ipython3 python3-scipy
 
   405  sudo apt install ipython3 python3-scipy
Line 147: Line 1,274:
 
   # installs to `~/.local/lib/python3.8/`  
 
   # installs to `~/.local/lib/python3.8/`  
 
#  
 
#  
−
   605  pip install --user wheel
+
   605  pip install --user   wheel
−
  606  pip install --user lightgbm
 
−
  630  pip install --user folium
 
 
   835  pip install --user --upgrade  --upgrade-strategy=only-if-needed pandas  
 
   835  pip install --user --upgrade  --upgrade-strategy=only-if-needed pandas  
 
   836  pip install --user --upgrade  --upgrade-strategy=only-if-needed scikit-image
 
   836  pip install --user --upgrade  --upgrade-strategy=only-if-needed scikit-image
 
   837  pip install --user --upgrade  --upgrade-strategy=only-if-needed imageio
 
   837  pip install --user --upgrade  --upgrade-strategy=only-if-needed imageio
 
   838  pip install --user --upgrade  --upgrade-strategy=only-if-needed numba
 
   838  pip install --user --upgrade  --upgrade-strategy=only-if-needed numba
 +
  630  pip install --user  folium
 
   839  pip install --user --upgrade  --upgrade-strategy=only-if-needed sentinelhub
 
   839  pip install --user --upgrade  --upgrade-strategy=only-if-needed sentinelhub
 +
  867  pip install --user --upgrade  --upgrade-strategy=only-if-needed openeo
 
   841  pip install --user --upgrade  --upgrade-strategy=only-if-needed eo-learn
 
   841  pip install --user --upgrade  --upgrade-strategy=only-if-needed eo-learn
 
   853  pip install --user --upgrade  --upgrade-strategy=only-if-needed ipyleaflet
 
   853  pip install --user --upgrade  --upgrade-strategy=only-if-needed ipyleaflet
−
   867 pip install --user --upgrade --upgrade-strategy=only-if-needed openeo
+
 
 +
   606 pip install --user   lightgbm ## libs also have an R interface, optional engines
  
 
all [ rasterio, networkx, sklearn, boto, more...  ] into  ~/.local/lib/python3.8/  (below)
 
all [ rasterio, networkx, sklearn, boto, more...  ] into  ~/.local/lib/python3.8/  (below)
 +
 +
    note: https://pip.pypa.io/en/stable/cli/pip_install/#hash-checking-mode
 +
 
</pre>
 
</pre>
  
Line 185: Line 1,316:
 
</pre>
 
</pre>
  
−
=== Focal APT Repos ===
+
 
 +
<pre>
 +
dbb@focalk:/etc$ cat lsb-release
 +
DISTRIB_ID=Ubuntu
 +
DISTRIB_RELEASE=20.04
 +
DISTRIB_CODENAME=focal
 +
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"
 +
#--
 +
libgeos3.10.1  (3.10.1-1~focal0)
 +
libgdal-dev    (3.4.0+dfsg-1~focal0)
 +
  postgresql-12-postgis-3    (3.2.0+dfsg-1~focal0)
 +
  PGSQL="120" GEOS="3.10.1-CAPI-1.16.0" PROJ="8.2.0" LIBXML="2.9.10" LIBJSON="0.13.1"
 +
#--
 +
</pre>
 +
 
 +
 
 +
=== geoxarray ===
 +
 
 
<pre>
 
<pre>
−
dbb@focalk:/etc/apt$ cat sources.list
+
$ pip install --user rioxarray
−
# Automatically generated by Calamares on 2022-01-02.
+
Collecting rioxarray
−
# Lines starting with "deb" are mandatory, while lines starting with "deb-src"
+
  Downloading rioxarray-0.10.3.tar.gz (47 kB)
−
# are for more detailed package information.
+
    |                              | 47 kB 272 kB/s
 +
  Installing build dependencies ... done
 +
  Getting requirements to build wheel ... done
 +
    Preparing wheel metadata ... done
 +
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from rioxarray) (20.3)
 +
Requirement already satisfied: xarray>=0.17 in ./.local/lib/python3.8/site-packages (from rioxarray) (0.21.1)
 +
Requirement already satisfied: rasterio>=1.1.1 in ./.local/lib/python3.8/site-packages (from rioxarray) (1.2.10)
 +
Requirement already satisfied: pyproj>=2.2 in /usr/lib/python3/dist-packages (from rioxarray) (3.2.1)
 +
Requirement already satisfied: pandas>=1.1 in ./.local/lib/python3.8/site-packages (from xarray>=0.17->rioxarray) (1.4.1)
 +
Requirement already satisfied: numpy>=1.18 in ./.local/lib/python3.8/site-packages (from xarray>=0.17->rioxarray) (1.21.5)
 +
Requirement already satisfied: click-plugins in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (1.1.1)
 +
Requirement already satisfied: cligj>=0.5 in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (0.5.0)
 +
Requirement already satisfied: attrs in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (19.3.0)
 +
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (2019.11.28)
 +
Requirement already satisfied: click>=4.0 in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (7.0)
 +
Requirement already satisfied: affine in ./.local/lib/python3.8/site-packages (from rasterio>=1.1.1->rioxarray) (2.3.0)
 +
Requirement already satisfied: snuggs>=1.4.1 in ./.local/lib/python3.8/site-packages (from rasterio>=1.1.1->rioxarray) (1.4.7)
 +
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (45.2.0)
 +
Requirement already satisfied: python-dateutil>=2.8.1 in ./.local/lib/python3.8/site-packages (from pandas>=1.1->xarray>=0.17->rioxarray) (2.8.2)
 +
Requirement already satisfied: pytz>=2020.1 in ./.local/lib/python3.8/site-packages (from pandas>=1.1->xarray>=0.17->rioxarray) (2021.3)
 +
Requirement already satisfied: pyparsing>=2.1.6 in /usr/lib/python3/dist-packages (from snuggs>=1.4.1->rasterio>=1.1.1->rioxarray) (2.4.6)
 +
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.1->pandas>=1.1->xarray>=0.17->rioxarray) (1.14.0)
 +
Building wheels for collected packages: rioxarray
 +
  Building wheel for rioxarray (PEP 517) ... done
 +
  Created wheel for rioxarray: filename=rioxarray-0.10.3-py3-none-any.whl size=55624 sha256=925199c830a54ca5e9b02e89060e822e27014db5cbe601fd6b513d106fe66687
 +
  Stored in directory: /home/dbb/.cache/pip/wheels/42/71/40/298b7df47153454b8eb3ef8e5441dafa413e5a2d7ac1832b17
 +
Successfully built rioxarray
 +
Installing collected packages: rioxarray
 +
Successfully installed rioxarray-0.10.3
 +
</pre>
  
−
## See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
 
−
## newer versions of Lubuntu.
 
−
deb http://ubuntu.osuosl.org/ubuntu/ focal main restricted
 
−
deb-src http://ubuntu.osuosl.org/ubuntu/ focal main restricted
 
  
−
## Major bug fix updates produced after the final release of Lubuntu.
+
== OSM software stack ==
−
## Have you noticed a regression? Please report it!
 
−
## https://wiki.ubuntu.com/StableReleaseUpdates#Regressions
 
  
−
## Software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu team.
+
'''osm2pgsql''' and '''libosmium'''
−
## Also, please note that software in Universe WILL NOT receive any review or
+
<pre>
−
## updates from the Ubuntu security team directly. Updates in this repository
 
−
## are provided by volunteers, but most come from Debian.
 
−
deb http://ubuntu.osuosl.org/ubuntu/ focal universe
 
−
deb-src http://ubuntu.osuosl.org/ubuntu/ focal universe
 
  
−
## Software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu team,
+
# jammy universe
−
## and may not be under a free licence. Please satisfy yourself as your rights
 
−
## to use the software. Also, please note that software in Multiverse WILL NOT
 
−
## receive any review or updates from the Ubuntu security team directly.
 
−
# deb http://us.archive.ubuntu.com/ubuntu/ focal multiverse
 
−
# deb-src http://us.archive.ubuntu.com/ubuntu/ focal multiverse
 
−
# deb http://us.archive.ubuntu.com/ubuntu/ focal-updates multiverse
 
−
# deb-src http://us.archive.ubuntu.com/ubuntu/ focal-updates multiverse
 
  
−
## Software from this repository contains tested security updates from the
+
    Thu May 26 07:55:03 PM PDT 2022
−
## Ubuntu security team.
 
−
deb http://security.ubuntu.com/ubuntu focal-security main restricted
 
−
deb-src http://security.ubuntu.com/ubuntu focal-security main restricted
 
−
deb http://security.ubuntu.com/ubuntu focal-security universe
 
−
deb-src http://security.ubuntu.com/ubuntu focal-security universe
 
−
deb http://security.ubuntu.com/ubuntu focal-security multiverse
 
−
# deb-src http://security.ubuntu.com/ubuntu focal-security multiverse
 
  
−
## Software from this repository may not have been tested as extensively as
+
osm2pgsql  1.6.0+ds-1
−
## software contained in the main release, although it includes newer versions
+
libosmium2-dev  Version: 2.18.0-1
−
## of some applications which may provide useful features. Also, please note
 
−
## that software in Backports WILL NOT receive any review or updates from the
 
−
## Ubuntu security team.
 
−
deb http://ubuntu.osuosl.org/ubuntu/ focal-backports main restricted universe multiverse
 
−
deb-src http://ubuntu.osuosl.org/ubuntu/ focal-backports main restricted universe multiverse
 
  
−
## Uncomment the following two lines to add software from Canonical's
+
Sat 26 Feb 2022 07:10:31 PM PST
−
## "partner" repository.
 
−
## This software is not part of Ubuntu, but is offered by Canonical and the
 
−
## respective vendors as a service to Ubuntu users.
 
−
# deb http://archive.canonical.com/ubuntu focal partner
 
−
# deb-src http://archive.canonical.com/ubuntu focal partner
 
  
−
-------------------------
+
# ubuntugis-unstable  focal
 +
osm2pgsql  1.3.0+ds-1~focal2
  
−
dbb@focalk:/etc/apt$ cat sources.list.d/ubuntugis-ubuntu-ubuntugis-unstable-focal.list
+
# git repo
 +
osm2pgsql  1.6    lonvia committed on Jan 24 2022
 +
osm2pgsql  1.5.1  Thu Jul 29 09:49:35 2021 +0200
  
−
deb http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main
+
# osm2pgsql cmake variables set
−
deb-src http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main
+
PROJ6_INCLUDE_DIR  /usr/include
 +
PROJ_LIBRARY  /usr/lib/x86_64-linux-gnu/libproj.so
 +
WITH_LUA  ON
 +
EXTERNAL_LIBOSMIUM  FALSE
 +
# from osm2pgsql/contrib/libosmium/
 +
LIBOSMIUM_VERSION_STRING "2.17.0"
  
 +
$ apt show libosmium2-dev  =>  2.17.2-1~focal0
  
−
</pre>
+
## the result of apt installing Universe  python3-pyosmium  2.15.3-1build2
 +
$ osmium --version  => osmium version 1.11.1, libosmium version 2.15.4
  
−
=== R setup ===
+
so, that is behind slightly from 2.17.x osmium headers
  
−
<pre>
+
## rebuild osm2pgsql 1.6.0,  better !
−
Focal default R
+
$  osm2pgsql --version
 +
2022-02-26 20:14:19  osm2pgsql version 1.6.0
 +
Build: Debug
 +
Compiled using the following library versions:
 +
Libosmium 2.17.3
 +
Proj [API 6] 8.2.0
 +
Lua 5.3.3
  
−
   $ R version 3.6.3 (2020-02-29) -- "Holding the Windsock"
+
$ sudo make install
 +
   ...
 +
[ 95%] Built target osm2pgsql_lib
 +
[100%] Built target osm2pgsql
 +
Install the project...
 +
-- Install configuration: "Debug"
 +
-- Installing: /usr/local/bin/osm2pgsql
 +
-- Installing: /usr/local/bin/osm2pgsql-replication
 +
-- Installing: /usr/local/share/man/man1/osm2pgsql-replication.1
 +
-- Installing: /usr/local/share/osm2pgsql/default.style
 +
-- Installing: /usr/local/share/osm2pgsql/empty.style
 +
-- Installing: /usr/local/share/man/man1/osm2pgsql.1
 
</pre>
 
</pre>
  
−
'''A better future''' awaits:  https://cran.r-project.org/bin/linux/ubuntu/
+
<pre>
 +
    OSMnx 
 +
      in tutorial  https://pygis.io/docs/d_access_osm.html
 +
      code https://pypi.org/project/osmnx/#files
 +
</pre>
  
−
--
 
−
R language example from aigeolabs.com
 
−
''Explainable Machine Learning for Land Cover Classification''
 
  
 
<pre>
 
<pre>
−
# Load the libraries
+
## osm2pgsql FLEX layout post-processing via LUA
−
rm(list=ls())
+
 
−
library(sf)    # represents simple features as native R objects
+
./flex-config/
−
library(rgdal) # interface to the “Geospatial Data Abstraction Library (GDAL)
+
  route-relations.lua  unitable.lua  geometries.lua
−
library(raster) # provides important functions for importing and handling raster data
+
  data-types.lua  attributes.lua  compatible.lua
−
library(rasterVis) # provides functions to display raster objects
+
   simple.lua  places.lua  generic.lua  with-schema.lua
−
library(dplyr) # provides basic data transformation functions
 
−
library(ggplot2) # provides extension for visualizations
 
−
library(mapview) # provides functions to visualize geospatial data
 
−
library(randomForest) # implements Breiman’s random forest (RF) algorithm
 
−
library(kernlab) # implements support vector machine (SVM) algorithm
 
−
library(corrplot) # provides graphical display of a correlation matrix
 
−
library(iml)   # Keras-base tools for analyzing machine learning models
 
−
library(caret) #  machine learning helper
 
  
−
##--------------
+
./tests/data/
 +
  test_output_flex_multigeom_polygon.lua
 +
  test_output_flex_way.lua
 +
  test_output_flex_area_3857.lua
 +
  test_output_flex_attr.lua
 +
  test_output_flex_stage2.lua
 +
  test_output_flex_nocluster.lua
 +
  test_output_flex_multigeom_multipolygon_split.lua
 +
  test_output_flex_multigeom.lua
 +
  test_output_flex_multigeom_multipolygon.lua
 +
  test_output_flex_nodes.lua
 +
  test_output_flex_area_4326.lua
 +
  test_output_flex_relation_combinations.lua
 +
  test_output_flex_nogeom.lua
 +
  test_output_flex_area.lua
 +
  test_output_flex_lua_fail.lua
 +
  test_output_flex_bbox.lua
 +
  test_output_flex_types.lua
 +
  test_output_flex_area_mix.lua
 +
  test_output_flex_relation_changes.lua
 +
  test_output_flex_schema.lua
 +
  test_output_flex_multigeom_geometry_split.lua
 +
  test_output_flex_stage2_alt.lua
 +
  test_output_flex_uni.lua
 +
  test_output_flex.lua
 +
  test_output_flex_validgeom.lua
 +
  test_output_flex_line.lua
 +
  test_output_flex_invalid_geom.lua
 +
  test_output_flex_multigeom_geometry.lua
 +
  test_output_flex_relations.lua
 
</pre>
 
</pre>
  
 
<pre>
 
<pre>
−
## Focal R apt
+
## a full list of osmium verbs
−
$ apt install r-base-core --yes
+
osmium-add-locations-to-ways   osmium-export                  osmium-merge-changes
−
  ...
+
osmium-apply-changes          osmium-extract                osmium-query-locations-index
−
The following NEW packages will be installed:
+
osmium-cat                    osmium-file-formats            osmium-renumber
−
   gfortran gfortran-9 libgfortran-9-dev libpcre2-32-0 libpcre2-dev libpcre2-posix2 r-base-core r-base-dev
+
osmium-changeset-filter        osmium-fileinfo                osmium-show
−
  r-cran-boot r-cran-class r-cran-cluster r-cran-codetools r-cran-foreign r-cran-kernsmooth r-cran-lattice
+
osmium-check-refs              osmium-getid                  osmium-sort
−
  r-cran-mass r-cran-matrix r-cran-mgcv r-cran-nlme r-cran-nnet r-cran-rpart r-cran-spatial r-cran-survival
+
osmium-create-locations-index  osmium-getparents              osmium-tags-filter
−
  r-doc-html r-recommended
+
osmium-derive-changes          osmium-index-types            osmium-time-filter
 +
osmium-diff                    osmium-merge                 
 +
</pre>
  
−
$ apt install r-cran-sf --yes
+
== Focal Jupyter Overview ==
−
  ...
 
−
The following NEW packages will be installed:
 
−
  fonts-glewlwyd libjs-d3 libjs-es5-shim libjs-highlight.js libjs-jquery-datatables
 
−
  libjs-jquery-datatables-extensions libjs-jquery-selectize.js libjs-json libjs-microplugin.js libjs-prettify
 
−
  libjs-sifter.js libjs-twitter-bootstrap-datepicker libnode64 libudunits2-0 libudunits2-data littler
 
−
  node-highlight.js node-html5shiv node-normalize.css nodejs nodejs-doc r-cran-abind r-cran-askpass
 
−
  r-cran-assertthat r-cran-backports r-cran-base64enc r-cran-bh r-cran-bit r-cran-bit64 r-cran-bitops r-cran-blob
 
−
  r-cran-brew r-cran-callr r-cran-classint r-cran-cli r-cran-cliapp r-cran-clipr r-cran-clisymbols
 
−
  r-cran-colorspace r-cran-commonmark r-cran-covr r-cran-crayon r-cran-crosstalk r-cran-curl r-cran-data.table
 
−
  r-cran-dbi r-cran-dbitest r-cran-deldir r-cran-desc r-cran-devtools r-cran-diffobj r-cran-digest r-cran-domc
 
−
  r-cran-dplyr r-cran-dt r-cran-e1071 r-cran-ellipsis r-cran-evaluate r-cran-fansi r-cran-farver r-cran-fastmap
 
−
  r-cran-filehash r-cran-fontbitstreamvera r-cran-fontliberation r-cran-fontquiver r-cran-foreach
 
−
  r-cran-freetypeharfbuzz r-cran-fs r-cran-future r-cran-gdtools r-cran-generics r-cran-ggplot2 r-cran-gh
 
−
  r-cran-git2r r-cran-globals r-cran-glue r-cran-goftest r-cran-gsl r-cran-gtable r-cran-hexbin r-cran-highr
 
−
  r-cran-hms r-cran-htmltools r-cran-htmlwidgets r-cran-httpuv r-cran-httr r-cran-ini r-cran-iterators
 
−
  r-cran-jsonlite r-cran-knitr r-cran-labeling r-cran-later r-cran-lazyeval r-cran-lifecycle r-cran-listenv
 
−
  r-cran-littler r-cran-lubridate r-cran-magrittr r-cran-mapproj r-cran-maps r-cran-maptools r-cran-markdown
 
−
  r-cran-memoise r-cran-mime r-cran-mockery r-cran-mockr r-cran-multicore r-cran-munsell r-cran-openssl
 
−
  r-cran-pillar r-cran-pkgbuild r-cran-pkgconfig r-cran-pkgkitten r-cran-pkgload r-cran-plogr r-cran-plyr
 
−
  r-cran-png r-cran-polyclip r-cran-praise r-cran-prettycode r-cran-prettyunits r-cran-processx r-cran-progress
 
−
  r-cran-promises r-cran-ps r-cran-purrr r-cran-r.methodss3 r-cran-r.oo r-cran-r6 r-cran-randomfields
 
−
  r-cran-randomfieldsutils r-cran-raster r-cran-rcmdcheck r-cran-rcolorbrewer r-cran-rcpp r-cran-remotes
 
−
  r-cran-reshape2 r-cran-rex r-cran-rgdal r-cran-rlang r-cran-rmarkdown r-cran-roxygen2 r-cran-rpostgresql
 
−
  r-cran-rprojroot r-cran-rsqlite r-cran-rstudioapi r-cran-runit r-cran-rversions r-cran-scales r-cran-selectr
 
−
  r-cran-sessioninfo r-cran-sf r-cran-shiny r-cran-sodium r-cran-sourcetools r-cran-sp r-cran-spatstat
 
−
  r-cran-spatstat.data r-cran-spatstat.utils r-cran-stringi r-cran-stringr r-cran-sys r-cran-systemfonts
 
−
  r-cran-tensor r-cran-testit r-cran-testthat r-cran-tibble r-cran-tidyr r-cran-tidyselect r-cran-tikzdevice
 
−
  r-cran-tinytex r-cran-units r-cran-usethis r-cran-utf8 r-cran-vctrs r-cran-vdiffr r-cran-viridislite
 
−
  r-cran-webutils r-cran-whisker r-cran-withr r-cran-xfun r-cran-xml r-cran-xml2 r-cran-xopen r-cran-xtable
 
−
  r-cran-yaml
 
−
0 upgraded, 186
 
−
</pre>
 
  
 
<pre>
 
<pre>
−
Warning from rgdal
+
                                    Sat 19 Feb 2022 07:18:48 PM PST
−
   ...
+
$ jupyter --version
−
> library(rgdal)
+
jupyter core    : 4.6.3
−
Loading required package: sp
+
jupyter-notebook : 6.0.3
−
rgdal: version: 1.5-18, (SVN revision 1082)
+
qtconsole        : not installed
−
Geospatial Data Abstraction Library extensions to R successfully loaded
+
ipython          : 7.13.0
−
Loaded GDAL runtime: GDAL 3.4.0, released 2021/11/04
+
ipykernel        : 5.2.0
−
Path to GDAL shared files: /usr/share/gdal
+
jupyter client   : 6.1.2
−
GDAL binary built with GEOS: TRUE
+
jupyter lab      : not installed
−
Loaded PROJ runtime: Rel. 8.2.0, November 1st, 2021, [PJ_VERSION: 820]
+
nbconvert        : 5.6.1
−
Path to PROJ shared files: /home/dbb/.local/share/proj:/usr/share/proj
+
ipywidgets      : 6.0.0
−
PROJ CDN enabled: FALSE
+
nbformat        : 5.0.4
−
Linking to sp version:1.4-0
+
traitlets        : 4.3.3
−
To mute warnings of possible GDAL/OSR exportToProj4() degradation,
+
 
−
use options("rgdal_show_exportToProj4_warnings"="none") before loading rgdal.
+
##-------------------------------
 +
Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
 +
pandas version: 0.25.3  <== a weak spot in this setup
 +
matplotlib version: 3.1.2
 +
NumPy version: 1.17.4
 +
SciPy version: 1.3.3
 +
IPython version: 7.13.0
 +
scikit-learn version: 1.0.2
 +
##-----------------------------------------
 +
dbb@focalk:~$ ipython3
 +
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
 +
Type 'copyright', 'credits' or 'license' for more information
 +
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.
 +
 
 +
In [1]: import sklearn                                                                                             
 +
In [2]: from sklearn import show_versions                                                                           
 +
In [3]: show_versions()                                                                                            
 +
 
 +
System:
 +
    python: 3.8.10 (default, Nov 26 2021, 20:14:08)  [GCC 9.3.0]
 +
executable: /usr/bin/python3
 +
  machine: Linux-5.13.0-30-generic-x86_64-with-glibc2.29
 +
 
 +
Python dependencies:
 +
          pip: 20.0.2
 +
  setuptools: 45.2.0
 +
      sklearn: 1.0.2
 +
        numpy: 1.17.4
 +
        scipy: 1.3.3
 +
      Cython: 0.29.14
 +
      pandas: 0.25.3      <== a weak spot in this setup
 +
  matplotlib: 3.1.2
 +
      joblib: 1.1.0
 +
threadpoolctl: 3.1.0
 +
 
 +
Built with OpenMP: True
 +
--
 +
dbb@focalk:~$ jupyter --paths
 +
config:
 +
    /home/dbb/.jupyter
 +
    /usr/local/etc/jupyter
 +
    /etc/jupyter
 +
data:
 +
    /home/dbb/.local/share/jupyter
 +
    /usr/local/share/jupyter
 +
    /usr/share/jupyter
 +
runtime:
 +
    /home/dbb/.local/share/jupyter/runtime
  
 
</pre>
 
</pre>
  
 +
=== Focal  APT  Repos ===
 
<pre>
 
<pre>
−
OSGeoLive install_R.sh ver14
+
dbb@focalk:/etc/apt$ cat sources.list
 +
# Automatically generated by Calamares on 2022-01-02.
 +
# Lines starting with "deb" are mandatory, while lines starting with "deb-src"
 +
# are for more detailed package information.
 +
...
 +
-------------------------
  
−
apt-get --assume-yes install r-base-core r-cran-sp r-cran-rpostgresql r-cran-udunits2 \
+
dbb@focalk:/etc/apt$ cat sources.list.d/ubuntugis-ubuntu-ubuntugis-unstable-focal.list
−
  r-cran-ggplot2 r-cran-sf r-cran-lwgeom r-cran-spdata r-cran-rgdal r-cran-maptools
 
  
 +
deb http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main
 +
deb-src http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main
 
</pre>
 
</pre>
  
Line 445: Line 1,661:
 
$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      sentinelhub
 
$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      sentinelhub
 
Requirement already up-to-date: sentinelhub in ./.local/lib/python3.8/site-packages (3.4.4)
 
Requirement already up-to-date: sentinelhub in ./.local/lib/python3.8/site-packages (3.4.4)
−
Requirement already satisfied, skipping upgrade: oauthlib in /usr/lib/python3/dist-packages (from sentinelhub) (3.1.0)
+
 
−
Requirement already satisfied, skipping upgrade: requests-oauthlib in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.3.1)
 
−
Requirement already satisfied, skipping upgrade: dataclasses-json in ./.local/lib/python3.8/site-packages (from sentinelhub) (0.5.6)
 
−
Requirement already satisfied, skipping upgrade: numpy in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.21.5)
 
−
Requirement already satisfied, skipping upgrade: pyproj>=2.2.0 in /usr/lib/python3/dist-packages (from sentinelhub) (3.2.1)
 
−
Requirement already satisfied, skipping upgrade: python-dateutil in ./.local/lib/python3.8/site-packages (from sentinelhub) (2.8.2)
 
−
Requirement already satisfied, skipping upgrade: requests>=2.5.0 in /usr/lib/python3/dist-packages (from sentinelhub) (2.22.0)
 
−
Requirement already satisfied, skipping upgrade: tifffile in ./.local/lib/python3.8/site-packages (from sentinelhub) (2022.2.9)
 
−
Requirement already satisfied, skipping upgrade: utm in ./.local/lib/python3.8/site-packages (from sentinelhub) (0.7.0)
 
 
Collecting pillow<=8.4.0,>=8.3.2
 
Collecting pillow<=8.4.0,>=8.3.2
 
   Downloading Pillow-8.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
 
   Downloading Pillow-8.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
 
     |                                | 3.1 MB 43 kB/s  
 
     |                                | 3.1 MB 43 kB/s  
−
Requirement already satisfied, skipping upgrade: click in /usr/lib/python3/dist-packages (from sentinelhub) (7.0)
+
 
−
Requirement already satisfied, skipping upgrade: aenum>=2.1.4 in ./.local/lib/python3.8/site-packages (from sentinelhub) (3.1.8)
 
−
Requirement already satisfied, skipping upgrade: boto3 in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.21.2)
 
−
Requirement already satisfied, skipping upgrade: shapely in /usr/lib/python3/dist-packages (from sentinelhub) (1.8.0)
 
−
Requirement already satisfied, skipping upgrade: botocore in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.24.2)
 
−
Requirement already satisfied, skipping upgrade: tqdm in ./.local/lib/python3.8/site-packages (from sentinelhub) (4.62.3)
 
−
Requirement already satisfied, skipping upgrade: typing-inspect>=0.4.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub) (0.7.1)
 
−
Requirement already satisfied, skipping upgrade: marshmallow-enum<2.0.0,>=1.5.1 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub) (1.5.1)
 
−
Requirement already satisfied, skipping upgrade: marshmallow<4.0.0,>=3.3.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub) (3.14.1)
 
−
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil->sentinelhub) (1.14.0)
 
−
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in ./.local/lib/python3.8/site-packages (from boto3->sentinelhub) (0.10.0)
 
−
Requirement already satisfied, skipping upgrade: s3transfer<0.6.0,>=0.5.0 in ./.local/lib/python3.8/site-packages (from boto3->sentinelhub) (0.5.1)
 
−
Requirement already satisfied, skipping upgrade: urllib3<1.27,>=1.25.4 in /usr/lib/python3/dist-packages (from botocore->sentinelhub) (1.25.8)
 
−
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub) (4.1.1)
 
−
Requirement already satisfied, skipping upgrade: mypy-extensions>=0.3.0 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub) (0.4.3)
 
 
Installing collected packages: pillow
 
Installing collected packages: pillow
 
   Attempting uninstall: pillow
 
   Attempting uninstall: pillow
Line 481: Line 1,675:
 
$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      eo-learn
 
$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      eo-learn
 
Requirement already up-to-date: eo-learn in ./.local/lib/python3.8/site-packages (1.0.0)
 
Requirement already up-to-date: eo-learn in ./.local/lib/python3.8/site-packages (1.0.0)
−
Requirement already satisfied, skipping upgrade: eo-learn-io>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: eo-learn-coregistration>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: eo-learn-features>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: eo-learn-mask>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: eo-learn-ml-tools>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: eo-learn-core>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: eo-learn-visualization>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: eo-learn-geometry>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
 
−
Requirement already satisfied, skipping upgrade: rasterio>=1.2.7 in ./.local/lib/python3.8/site-packages (from eo-learn-io>=1.0.0->eo-learn) (1.2.10)
 
−
Requirement already satisfied, skipping upgrade: sentinelhub>=3.4.0 in ./.local/lib/python3.8/site-packages (from eo-learn-io>=1.0.0->eo-learn) (3.4.4)
 
−
Requirement already satisfied, skipping upgrade: fiona>=1.8.18 in /usr/lib/python3/dist-packages (from eo-learn-io>=1.0.0->eo-learn) (1.8.20)
 
−
Requirement already satisfied, skipping upgrade: rtree in /usr/lib/python3/dist-packages (from eo-learn-io>=1.0.0->eo-learn) (0.9.4)
 
−
Requirement already satisfied, skipping upgrade: geopandas>=0.8.1 in /usr/lib/python3/dist-packages (from eo-learn-io>=1.0.0->eo-learn) (0.10.2)
 
−
Requirement already satisfied, skipping upgrade: boto3 in ./.local/lib/python3.8/site-packages (from eo-learn-io>=1.0.0->eo-learn) (1.21.2)
 
−
Requirement already satisfied, skipping upgrade: opencv-contrib-python-headless>=4.1 in ./.local/lib/python3.8/site-packages (from eo-learn-coregistration>=1.0.0->eo-learn) (4.5.5.62)
 
−
Requirement already satisfied, skipping upgrade: thunder-registration in ./.local/lib/python3.8/site-packages (from eo-learn-coregistration>=1.0.0->eo-learn) (1.0.1)
 
−
Requirement already satisfied, skipping upgrade: python-dateutil in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (2.8.2)
 
−
Requirement already satisfied, skipping upgrade: scipy in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (1.8.0)
 
−
Requirement already satisfied, skipping upgrade: numba>=0.53.0 in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (0.55.1)
 
−
Requirement already satisfied, skipping upgrade: numpy in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (1.21.5)
 
−
Requirement already satisfied, skipping upgrade: scikit-learn in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (1.0.2)
 
−
Requirement already satisfied, skipping upgrade: scikit-image>=0.19.0 in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (0.19.2)
 
−
Requirement already satisfied, skipping upgrade: lightgbm>=2.0.11 in ./.local/lib/python3.8/site-packages (from eo-learn-mask>=1.0.0->eo-learn) (3.3.2)
 
−
Requirement already satisfied, skipping upgrade: shapely in /usr/lib/python3/dist-packages (from eo-learn-ml-tools>=1.0.0->eo-learn) (1.8.0)
 
−
Requirement already satisfied, skipping upgrade: attrs>=19.2.0 in /usr/lib/python3/dist-packages (from eo-learn-core>=1.0.0->eo-learn) (19.3.0)
 
−
Requirement already satisfied, skipping upgrade: tqdm>=4.27 in ./.local/lib/python3.8/site-packages (from eo-learn-core>=1.0.0->eo-learn) (4.62.3)
 
−
Requirement already satisfied, skipping upgrade: fs in ./.local/lib/python3.8/site-packages (from eo-learn-core>=1.0.0->eo-learn) (2.4.15)
 
−
Requirement already satisfied, skipping upgrade: fs-s3fs in ./.local/lib/python3.8/site-packages (from eo-learn-core>=1.0.0->eo-learn) (1.1.1)
 
−
Requirement already satisfied, skipping upgrade: pydot in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (1.4.1)
 
−
Requirement already satisfied, skipping upgrade: jinja2 in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (2.10.1)
 
−
Requirement already satisfied, skipping upgrade: matplotlib in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (3.1.2)
 
−
Requirement already satisfied, skipping upgrade: graphviz>=0.10.1 in ./.local/lib/python3.8/site-packages (from eo-learn-visualization>=1.0.0->eo-learn) (0.19.1)
 
−
Requirement already satisfied, skipping upgrade: pygments in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (2.3.1)
 
−
Requirement already satisfied, skipping upgrade: descartes in /usr/lib/python3/dist-packages (from eo-learn-geometry>=1.0.0->eo-learn) (1.1.0)
 
−
Requirement already satisfied, skipping upgrade: certifi in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (2019.11.28)
 
−
Requirement already satisfied, skipping upgrade: setuptools in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (45.2.0)
 
−
Requirement already satisfied, skipping upgrade: click-plugins in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (1.1.1)
 
−
Requirement already satisfied, skipping upgrade: cligj>=0.5 in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (0.5.0)
 
−
Requirement already satisfied, skipping upgrade: affine in ./.local/lib/python3.8/site-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (2.3.0)
 
−
Requirement already satisfied, skipping upgrade: snuggs>=1.4.1 in ./.local/lib/python3.8/site-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (1.4.7)
 
−
Requirement already satisfied, skipping upgrade: click>=4.0 in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (7.0)
 
−
Requirement already satisfied, skipping upgrade: pyproj>=2.2.0 in /usr/lib/python3/dist-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.2.1)
 
−
Requirement already satisfied, skipping upgrade: oauthlib in /usr/lib/python3/dist-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.1.0)
 
−
Requirement already satisfied, skipping upgrade: aenum>=2.1.4 in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.1.8)
 
−
Requirement already satisfied, skipping upgrade: dataclasses-json in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.5.6)
 
−
Requirement already satisfied, skipping upgrade: requests>=2.5.0 in /usr/lib/python3/dist-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (2.22.0)
 
−
Requirement already satisfied, skipping upgrade: tifffile in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (2022.2.9)
 
−
Requirement already satisfied, skipping upgrade: utm in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.7.0)
 
−
Requirement already satisfied, skipping upgrade: requests-oauthlib in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.3.1)
 
−
Requirement already satisfied, skipping upgrade: pillow<=8.4.0,>=8.3.2 in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (8.4.0)
 
−
Requirement already satisfied, skipping upgrade: botocore in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.24.2)
 
−
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in ./.local/lib/python3.8/site-packages (from boto3->eo-learn-io>=1.0.0->eo-learn) (0.10.0)
 
−
Requirement already satisfied, skipping upgrade: s3transfer<0.6.0,>=0.5.0 in ./.local/lib/python3.8/site-packages (from boto3->eo-learn-io>=1.0.0->eo-learn) (0.5.1)
 
−
Requirement already satisfied, skipping upgrade: thunder-python>=1.2.0 in ./.local/lib/python3.8/site-packages (from thunder-registration->eo-learn-coregistration>=1.0.0->eo-learn) (1.4.2)
 
−
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil->eo-learn-features>=1.0.0->eo-learn) (1.14.0)
 
−
Requirement already satisfied, skipping upgrade: llvmlite<0.39,>=0.38.0rc1 in ./.local/lib/python3.8/site-packages (from numba>=0.53.0->eo-learn-features>=1.0.0->eo-learn) (0.38.0)
 
−
Requirement already satisfied, skipping upgrade: threadpoolctl>=2.0.0 in ./.local/lib/python3.8/site-packages (from scikit-learn->eo-learn-features>=1.0.0->eo-learn) (3.1.0)
 
−
Requirement already satisfied, skipping upgrade: joblib>=0.11 in ./.local/lib/python3.8/site-packages (from scikit-learn->eo-learn-features>=1.0.0->eo-learn) (1.1.0)
 
−
Requirement already satisfied, skipping upgrade: PyWavelets>=1.1.1 in ./.local/lib/python3.8/site-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (1.2.0)
 
−
Requirement already satisfied, skipping upgrade: imageio>=2.4.1 in ./.local/lib/python3.8/site-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (2.16.0)
 
−
Requirement already satisfied, skipping upgrade: networkx>=2.2 in ./.local/lib/python3.8/site-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (2.6.3)
 
−
Requirement already satisfied, skipping upgrade: packaging>=20.0 in /usr/lib/python3/dist-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (20.3)
 
−
Requirement already satisfied, skipping upgrade: wheel in /usr/lib/python3/dist-packages (from lightgbm>=2.0.11->eo-learn-mask>=1.0.0->eo-learn) (0.34.2)
 
−
Requirement already satisfied, skipping upgrade: pytz in ./.local/lib/python3.8/site-packages (from fs->eo-learn-core>=1.0.0->eo-learn) (2021.3)
 
−
Requirement already satisfied, skipping upgrade: appdirs~=1.4.3 in ./.local/lib/python3.8/site-packages (from fs->eo-learn-core>=1.0.0->eo-learn) (1.4.4)
 
−
Requirement already satisfied, skipping upgrade: pyparsing>=2.1.6 in /usr/lib/python3/dist-packages (from snuggs>=1.4.1->rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (2.4.6)
 
−
Requirement already satisfied, skipping upgrade: typing-inspect>=0.4.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.7.1)
 
−
Requirement already satisfied, skipping upgrade: marshmallow<4.0.0,>=3.3.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.14.1)
 
−
Requirement already satisfied, skipping upgrade: marshmallow-enum<2.0.0,>=1.5.1 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.5.1)
 
−
Requirement already satisfied, skipping upgrade: urllib3<1.27,>=1.25.4 in /usr/lib/python3/dist-packages (from botocore->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.25.8)
 
−
Requirement already satisfied, skipping upgrade: boto>=2.36.0 in ./.local/lib/python3.8/site-packages (from thunder-python>=1.2.0->thunder-registration->eo-learn-coregistration>=1.0.0->eo-learn) (2.49.0)
 
−
Requirement already satisfied, skipping upgrade: bolt-python>=0.7.0 in ./.local/lib/python3.8/site-packages (from thunder-python>=1.2.0->thunder-registration->eo-learn-coregistration>=1.0.0->eo-learn) (0.7.1)
 
−
Requirement already satisfied, skipping upgrade: mypy-extensions>=0.3.0 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.4.3)
 
−
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (4.1.1)
 
  
 
$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      ipyleaflet
 
$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      ipyleaflet
Line 585: Line 1,705:
 
##jupyter nbextension enable --py --sys-prefix{?} ipyleaflet  # can be skipped for notebook 5.3 and above
 
##jupyter nbextension enable --py --sys-prefix{?} ipyleaflet  # can be skipped for notebook 5.3 and above
 
</pre>
 
</pre>
 +
 +
== R setup ==
  
 
<pre>
 
<pre>
−
## snapshot local python install -- all packages
+
 
−
                                    Sat 26 Feb 2022 07:10:31 PM PST
+
--
 +
R language example
 +
 
 +
''Explainable Machine Learning for Land Cover Classification'' by Courage K.  2021
 +
 
 +
<pre>
 +
# Load the libraries example code from aigeolabs.com
 +
rm(list=ls())
 +
library(sf)    # represents simple features as native R objects
 +
library(rgdal) # interface to the “Geospatial Data Abstraction Library (GDAL)
 +
library(raster) # provides important functions for importing and handling raster data
 +
library(rasterVis) # provides functions to display raster objects
 +
library(dplyr) # provides basic data transformation functions
 +
library(ggplot2) # provides extension for visualizations
 +
library(mapview) # provides functions to visualize geospatial data
 +
library(randomForest) # implements Breiman’s random forest (RF) algorithm
 +
library(kernlab) # implements support vector machine (SVM) algorithm
 +
library(corrplot) # provides graphical display of a correlation matrix
 +
library(iml)  # Keras-base tools for analyzing machine learning models
 +
library(caret) #  machine learning helper
 +
 
 +
##--------------
 +
</pre>
 +
 
 +
 
 +
<pre>
 +
OSGeoLive install_R.sh ver14
 +
 
 +
apt-get --assume-yes install r-base-core r-cran-sp r-cran-rpostgresql r-cran-udunits2 \
 +
  r-cran-ggplot2 r-cran-sf r-cran-lwgeom r-cran-spdata r-cran-rgdal r-cran-maptools
 +
 
 +
</pre>
 +
 
 +
 
 +
 
 +
== osgeolive 10 patch ==
 +
in 2016, after being alerted [https://blog.jupyter.org/2016/09/26/ipywidget-security-release/ -LINK-]  , I used <code>pip</code> to upgrade <strong>ipywidgets</strong> on the OSGeo-Live 10.0 base system.
 +
 
 +
The following is a modified patch file showing the changes in Python 2.7:
 +
<pre>
 +
 
 +
## Major components, version changed
 +
-ipykernel==4.2.2
 +
+ipykernel==4.5.2
 +
...
 +
-ipython==4.0.2
 +
+ipython==5.1.0
 +
-ipywidgets==5.1.5
 +
+ipywidgets==5.2.2
 +
-jupyter-client==4.1.1
 +
+jupyter-client==4.4.0
 +
jupyter-console==4.1.0
 +
-jupyter-core==4.0.6
 +
+jupyter-core==4.2.1
 +
-notebook==4.2.1
 +
+notebook==4.3.0
 +
-nbconvert==4.1.0
 +
-nbformat==4.0.1
 +
+nbconvert==5.0.0
 +
+nbformat==4.2.0
 +
+html5lib==0.9999999
 +
-ptyprocess==0.5
 +
+ptyprocess==0.5.1
 +
-pexpect==4.0.1
 +
-pickleshare==0.5
 +
+pexpect==4.2.1
 +
+pickleshare==0.7.4
 +
 
 +
-decorator==4.0.6
 +
+decorator==4.0.10
 +
 
 +
##-- misc other additions --
 +
+backports-abc==0.5
 +
+backports.shutil-get-terminal-size==1.0.0
 +
+bleach==1.5.0
 +
+certifi==2016.9.26
 +
+configparser==3.5.0
 +
-enum34==1.1.2
 +
+entrypoints==0.2.2
 +
+enum34==1.1.6
 +
-mistune==0.7.1
 +
+mistune==0.7.3
 +
+pandocfilters==1.4.1
 +
+pathlib2==2.1.0
 +
+prompt-toolkit==1.0.9
 +
-Pygments==2.1
 +
+Pygments==2.1.3
 +
-pyzmq==15.2.0
 +
+pyzmq==16.0.2
 +
+testpath==0.3
 +
-tornado==4.2.1
 +
-traitlets==4.1.0
 +
+tornado==4.4.2
 +
+traitlets==4.3.1
 +
+wcwidth==0.1.7
 +
-widgetsnbextension==1.2.3
 +
+widgetsnbextension==1.2.6
 +
 
 +
##-- misc other python libs (unchanged)
 +
anyjson==0.3.3
 +
Biggus==0.13.0
 +
Cartopy==0.14.2
 +
celery==3.1.20
 +
cf-units==1.0.0
 +
colorama==0.3.7
 +
cycler==0.9.0
 +
defer==1.0.6
 +
descartes==1.0.1
 +
filemagic==1.6
 +
Fiona==1.6.3
 +
Iris==1.9.2
 +
imposm==2.6.0
 +
imposm.parser==1.0.7
 +
ipyleaflet==0.2.0
 +
jsonschema==2.5.1
 +
meld==3.14.2
 +
ModestMaps==1.4.6
 +
netCDF4==1.2.4
 +
Nik4==1.6.0
 +
numpy==1.11.0
 +
osmapi==0.6.2
 +
OWSLib==0.10.3
 +
pandas==0.17.1
 +
psutil==3.4.2
 +
psycopg2==2.6.1
 +
pykml==0.1.1
 +
pywps==3.2.5
 +
rasterio==0.31.0
 +
TileStache==1.49.8
 +
</pre>
 +
 
 +
== Python Packages All ==
 +
=== jammy Snapshot python install ===
 +
 
 +
<pre>
 +
        osgeolive rc1 python3_all
 +
 
 +
APScheduler==0.0.0
 +
Automat==20.2.0
 +
Babel==2.8.0
 +
Brotli==1.0.9
 +
Cartopy==0.0.0
 +
Cheetah3==3.2.6
 +
Deprecated==1.2.13
 +
Django==2.2.24
 +
EOxServer==1.1.2
 +
Faker==0.9.3
 +
Fiona==1.8.21
 +
Flask==2.0.1
 +
GDAL==3.4.3
 +
GeoAlchemy2==0.10.2
 +
GeoNode==3.3.2.post2
 +
HeapDict==1.0.1
 +
Jinja2==3.0.3
 +
Mako==1.1.3
 +
MapProxy==1.15.1
 +
Markdown==3.3.6
 +
MarkupSafe==2.0.1
 +
Nik4==1.7.0
 +
OWSLib==0.25.0
 +
Pillow==9.0.1
 +
PyGObject==3.42.1
 +
PyHamcrest==2.0.2
 +
PyJWT==2.3.0
 +
PyNaCl==1.5.0
 +
PyOpenGL==3.1.5
 +
PyQt5-sip==12.9.1
 +
PyQt5==5.15.6
 +
PyYAML==5.4.1
 +
Pygments==2.11.2
 +
QScintilla==2.11.6
 +
Rtree==0.9.7
 +
SQLAlchemy==1.4.31
 +
SecretStorage==3.3.1
 +
Send2Trash==1.8.1b0
 +
Shapely==1.8.0
 +
Twisted==22.1.0
 +
Unidecode==1.3.3
 +
Werkzeug==2.0.2
 +
adal==1.2.2
 +
affine==2.3.0
 +
aiohttp==3.8.1
 +
aioredis==1.3.1
 +
aiosignal==1.2.0
 +
amqp==5.0.9
 +
anyjson==0.3.3
 +
appdirs==1.4.4
 +
apt-xapian-index==0.49
 +
argon2-cffi==21.1.0
 +
async-timeout==4.0.1
 +
attrs==21.2.0
 +
avro==1.11.0
 +
azure-agrifood-farming==1.0.0b1
 +
azure-ai-anomalydetector==3.0.0b3
 +
azure-ai-formrecognizer==3.2.0b2
 +
azure-ai-language-conversations==1.0.0b1
 +
azure-ai-language-questionanswering==1.0.0
 +
azure-ai-metricsadvisor==1.0.1
 +
azure-ai-textanalytics==5.2.0b3
 +
azure-ai-translation-document==1.0.0b6
 +
azure-appconfiguration==1.3.0
 +
azure-applicationinsights==0.1.0
 +
azure-batch==11.0.0
 +
azure-cognitiveservices-anomalydetector==0.3.0
 +
azure-cognitiveservices-formrecognizer==0.1.1
 +
azure-cognitiveservices-knowledge-qnamaker==0.3.0
 +
azure-cognitiveservices-language-luis==0.7.0
 +
azure-cognitiveservices-language-spellcheck==2.0.0
 +
azure-cognitiveservices-language-textanalytics==0.2.0
 +
azure-cognitiveservices-personalizer==0.1.0
 +
azure-cognitiveservices-search-autosuggest==0.2.0
 +
azure-cognitiveservices-search-customimagesearch==0.2.0
 +
azure-cognitiveservices-search-customsearch==0.3.0
 +
azure-cognitiveservices-search-entitysearch==2.0.0
 +
azure-cognitiveservices-search-imagesearch==2.0.0
 +
azure-cognitiveservices-search-newssearch==2.0.0
 +
azure-cognitiveservices-search-videosearch==2.0.0
 +
azure-cognitiveservices-search-visualsearch==0.2.0
 +
azure-cognitiveservices-search-websearch==2.0.0
 +
azure-cognitiveservices-vision-computervision==0.9.0
 +
azure-cognitiveservices-vision-contentmoderator==1.0.0
 +
azure-cognitiveservices-vision-customvision==3.1.0
 +
azure-cognitiveservices-vision-face==0.5.0
 +
azure-common==1.1.28
 +
azure-communication-chat==1.1.0
 +
azure-communication-identity==1.0.1
 +
azure-communication-networktraversal==1.0.0b1
 +
azure-communication-phonenumbers==1.0.1
 +
azure-communication-sms==1.0.1
 +
azure-confidentialledger==1.0.0b1
 +
azure-containerregistry==1.0.0b7
 +
azure-core==1.20.1
 +
azure-data-tables==12.1.1
 +
azure-digitaltwins-core==1.1.0
 +
azure-eventgrid==4.7.0
 +
azure-eventhub-checkpointstoreblob-aio==1.1.4
 +
azure-eventhub-checkpointstoreblob==1.1.4
 +
azure-eventhub-checkpointstoretable==1.0.0b1
 +
azure-eventhub==5.6.2
 +
azure-graphrbac==0.61.1
 +
azure-identity==1.7.1
 +
azure-iot-deviceupdate==1.0.0b1
 +
azure-iot-modelsrepository==1.0.0b2
 +
azure-keyvault-administration==4.1.0b2
 +
azure-keyvault-certificates==4.4.0b2
 +
azure-keyvault-keys==4.5.0b5
 +
azure-keyvault-secrets==4.4.0b2
 +
azure-keyvault==4.1.0
 +
azure-loganalytics==0.1.0
 +
azure-media-analytics-edge==1.0.0b1
 +
azure-media-videoanalyzer-edge==1.0.0b3
 +
azure-messaging-webpubsubservice==1.0.0b3
 +
azure-mgmt-advisor==9.0.0
 +
azure-mgmt-agfood==1.0.0b1
 +
azure-mgmt-agrifood==1.0.0b1
 +
azure-mgmt-alertsmanagement==1.0.0
 +
azure-mgmt-apimanagement==2.1.0
 +
azure-mgmt-appconfiguration==2.0.0
 +
azure-mgmt-applicationinsights==1.0.0
 +
azure-mgmt-appplatform==6.1.0
 +
azure-mgmt-attestation==1.0.0
 +
azure-mgmt-authorization==2.0.0
 +
azure-mgmt-automanage==1.0.0b2
 +
azure-mgmt-automation==1.1.0b1
 +
azure-mgmt-avs==7.0.0b1
 +
azure-mgmt-azureadb2c==1.0.0b1
 +
azure-mgmt-azurearcdata==1.0.0
 +
azure-mgmt-azurestack==1.0.0
 +
azure-mgmt-azurestackhci==6.1.0b1
 +
azure-mgmt-baremetalinfrastructure==1.0.0
 +
azure-mgmt-batch==16.0.0
 +
azure-mgmt-batchai==7.0.0b1
 +
azure-mgmt-billing==6.0.0
 +
azure-mgmt-botservice==1.0.0
 +
azure-mgmt-cdn==11.0.0
 +
azure-mgmt-changeanalysis==1.0.0
 +
azure-mgmt-chaos==1.0.0b2
 +
azure-mgmt-cognitiveservices==12.0.0
 +
azure-mgmt-commerce==6.0.0
 +
azure-mgmt-communication==1.0.0
 +
azure-mgmt-compute==23.1.0
 +
azure-mgmt-confidentialledger==1.0.0b1
 +
azure-mgmt-confluent==2.0.0b1
 +
azure-mgmt-consumption==8.0.0
 +
azure-mgmt-containerinstance==9.1.0
 +
azure-mgmt-containerregistry==8.2.0
 +
azure-mgmt-containerservice==16.3.0
 +
azure-mgmt-core==1.3.0
 +
azure-mgmt-cosmosdb==7.0.0b2
 +
azure-mgmt-costmanagement==3.0.0
 +
azure-mgmt-customproviders==1.0.0
 +
azure-mgmt-databox==1.0.0
 +
azure-mgmt-databoxedge==1.0.0
 +
azure-mgmt-databricks==1.1.0b1
 +
azure-mgmt-datadog==2.0.0
 +
azure-mgmt-datafactory==2.0.0
 +
azure-mgmt-datalake-analytics==0.6.0
 +
azure-mgmt-datalake-store==1.0.0
 +
azure-mgmt-datamigration==10.0.0
 +
azure-mgmt-dataprotection==1.0.0b1
 +
azure-mgmt-datashare==1.0.0
 +
azure-mgmt-deploymentmanager==1.0.0
 +
azure-mgmt-deviceupdate==1.0.0b3
 +
azure-mgmt-devspaces==1.0.0b1
 +
azure-mgmt-devtestlabs==9.0.0
 +
azure-mgmt-digitaltwins==6.0.0
 +
azure-mgmt-dns==8.0.0
 +
azure-mgmt-documentdb==0.1.3
 +
azure-mgmt-edgegateway==0.1.0
 +
azure-mgmt-edgeorder==1.0.0b1
 +
azure-mgmt-elastic==1.0.0
 +
azure-mgmt-eventgrid==9.0.0
 +
azure-mgmt-eventhub==10.0.0
 +
azure-mgmt-extendedlocation==1.0.0
 +
azure-mgmt-fluidrelay==1.0.0b1
 +
azure-mgmt-frontdoor==1.0.0
 +
azure-mgmt-guestconfig==1.0.0b1
 +
azure-mgmt-hanaonazure==1.0.0
 +
azure-mgmt-hdinsight==9.0.0
 +
azure-mgmt-healthbot==1.0.0b1
 +
azure-mgmt-healthcareapis==1.1.0b1
 +
azure-mgmt-hybridcompute==7.0.0
 +
azure-mgmt-hybridkubernetes==1.1.0
 +
azure-mgmt-hybridnetwork==1.0.0
 +
azure-mgmt-imagebuilder==1.0.0b1
 +
azure-mgmt-iotcentral==9.0.0b1
 +
azure-mgmt-iothub==2.1.0
 +
azure-mgmt-iothubprovisioningservices==1.0.0
 +
azure-mgmt-keyvault==9.2.0
 +
azure-mgmt-kubernetesconfiguration==1.0.0b1
 +
azure-mgmt-kusto==2.1.0
 +
azure-mgmt-labservices==1.0.0
 +
azure-mgmt-loganalytics==11.0.0
 +
azure-mgmt-logic==9.0.0
 +
azure-mgmt-logz==1.0.0
 +
azure-mgmt-machinelearningcompute==1.0.0b1
 +
azure-mgmt-machinelearningservices==1.0.0
 +
azure-mgmt-maintenance==2.0.0
 +
azure-mgmt-managedservices==6.0.0
 +
azure-mgmt-managementgroups==1.0.0
 +
azure-mgmt-managementpartner==1.0.0
 +
azure-mgmt-maps==2.0.0
 +
azure-mgmt-marketplaceordering==1.1.0
 +
azure-mgmt-media==8.0.0
 +
azure-mgmt-mixedreality==1.0.0
 +
azure-mgmt-monitor==2.0.0
 +
azure-mgmt-msi==6.0.0b1
 +
azure-mgmt-netapp==5.1.0
 +
azure-mgmt-network==19.2.0
 +
azure-mgmt-notificationhubs==7.0.0
 +
azure-mgmt-operationsmanagement==1.0.0
 +
azure-mgmt-peering==1.0.0
 +
azure-mgmt-policyinsights==1.1.0b1
 +
azure-mgmt-portal==1.0.0
 +
azure-mgmt-powerbidedicated==1.0.0
 +
azure-mgmt-powerbiembedded==2.0.0
 +
azure-mgmt-privatedns==1.0.0
 +
azure-mgmt-purview==1.0.0
 +
azure-mgmt-quantum==1.0.0b2
 +
azure-mgmt-quota==1.0.0b2
 +
azure-mgmt-rdbms==10.0.0
 +
azure-mgmt-recoveryservices==2.0.0
 +
azure-mgmt-recoveryservicesbackup==3.0.0
 +
azure-mgmt-recoveryservicessiterecovery==1.0.0b1
 +
azure-mgmt-redhatopenshift==1.0.0
 +
azure-mgmt-redis==13.0.0
 +
azure-mgmt-redisenterprise==1.0.0
 +
azure-mgmt-regionmove==1.0.0b1
 +
azure-mgmt-relay==1.1.0
 +
azure-mgmt-reservations==1.0.0
 +
azure-mgmt-resource==20.0.0
 +
azure-mgmt-resourcegraph==8.1.0b1
 +
azure-mgmt-resourcehealth==1.0.0b1
 +
azure-mgmt-resourcemover==1.1.0b2
 +
azure-mgmt-scheduler==7.0.0b1
 +
azure-mgmt-search==8.0.0
 +
azure-mgmt-security==2.0.0b1
 +
azure-mgmt-securityinsight==1.0.0b1
 +
azure-mgmt-serialconsole==1.0.0
 +
azure-mgmt-servermanager==2.0.0
 +
azure-mgmt-servicebus==7.1.0
 +
azure-mgmt-servicefabric==2.0.0
 +
azure-mgmt-servicefabricmanagedclusters==1.0.0
 +
azure-mgmt-servicelinker==1.0.0b1
 +
azure-mgmt-signalr==1.0.0
 +
azure-mgmt-sql==3.0.1
 +
azure-mgmt-sqlvirtualmachine==1.0.0b1
 +
azure-mgmt-storage==19.0.0
 +
azure-mgmt-storagecache==1.1.0
 +
azure-mgmt-storageimportexport==1.0.0b1
 +
azure-mgmt-storagepool==1.0.0
 +
azure-mgmt-storagesync==1.0.0
 +
azure-mgmt-streamanalytics==1.0.0rc1
 +
azure-mgmt-subscription==1.0.0
 +
azure-mgmt-support==6.0.0
 +
azure-mgmt-synapse==2.1.0b2
 +
azure-mgmt-testbase==1.0.0b1
 +
azure-mgmt-timeseriesinsights==1.0.0
 +
azure-mgmt-trafficmanager==1.0.0b1
 +
azure-mgmt-videoanalyzer==1.0.0b3
 +
azure-mgmt-vmwarecloudsimple==1.0.0b1
 +
azure-mgmt-web==5.0.0
 +
azure-mgmt-webpubsub==1.0.0
 +
azure-mgmt-workloadmonitor==1.0.0b2
 +
azure-mixedreality-authentication==1.0.0b1
 +
azure-monitor-opentelemetry-exporter==1.0.0b6
 +
azure-monitor-query==1.0.1
 +
azure-purview-administration==1.0.0b2
 +
azure-purview-catalog==1.0.0b3
 +
azure-purview-scanning==1.0.0b3
 +
azure-schemaregistry-avroserializer==1.0.0b4
 +
azure-schemaregistry==1.0.0
 +
azure-search-documents==11.3.0b5
 +
azure-security-attestation==1.0.1
 +
azure-servicebus==7.4.0
 +
azure-servicefabric==8.0.0.0
 +
azure-servicemanagement-legacy==0.20.7
 +
azure-storage-blob-changefeed==12.0.0b3
 +
azure-storage-blob==12.9.0
 +
azure-storage-file-datalake==12.5.0
 +
azure-storage-file-share==12.6.0
 +
azure-storage-queue==12.1.6
 +
azure-synapse-accesscontrol==0.8.0
 +
azure-synapse-artifacts==0.10.0
 +
azure-synapse-managedprivateendpoints==0.5.0
 +
azure-synapse-monitoring==0.3.0
 +
azure-synapse-spark==0.8.0
 +
azure-synapse==0.1.1
 +
azure-template==0.0.18b3
 +
backcall==0.2.0
 +
bcrypt==3.2.0
 +
beautifulsoup4==4.10.0
 +
billiard==3.6.4.0
 +
bleach==4.1.0
 +
blinker==1.4
 +
boto3==1.20.34
 +
botocore==1.23.34
 +
branca==0.4.2
 +
cachetools==5.0.0
 +
celery==5.2.3
 +
certifi==2020.6.20
 +
cftime==1.5.2
 +
chardet==4.0.0
 +
charset-normalizer==2.0.6
 +
click-didyoumean==0.0.3
 +
click-plugins==1.1.1
 +
click-repl==0.2.0
 +
click==8.0.3
 +
cligj==0.7.2
 +
cloudpickle==2.0.0
 +
colorama==0.4.4
 +
command-not-found==0.3
 +
constantly==15.1.0
 +
coverage==6.2
 +
croniter==1.0.15
 +
cryptography==3.4.8
 +
cupshelpers==1.0
 +
cycler==0.11.0
 +
dask==2022.1.0+dfsg
 +
datacube==0.0.0
 +
dbus-python==1.2.18
 +
decorator==4.4.2
 +
defer==1.0.6
 +
defusedxml==0.7.1
 +
deprecation==2.0.7
 +
descartes==1.1.0
 +
distributed==2022.1.0+ds.1
 +
distro-info===1.1build1
 +
distro==1.7.0
 +
dj-database-url==0.5.0
 +
dj-pagination==2.5.0
 +
django-allauth==0.47.0
 +
django-appconf==1.0.3
 +
django-autocomplete-light==3.5.1
 +
django-bootstrap3-datetimepicker-2==2.8.2
 +
django-celery-beat==2.2.1
 +
django-celery-results==2.0.0
 +
django-cors-headers==3.11.0
 +
django-downloadview==2.3.0
 +
django-filter==2.4.0
 +
django-floppyforms==1.9.0
 +
django-forms-bootstrap==3.1.0
 +
django-geonode-mapstore-client==3.3.2
 +
django-grappelli==2.15.1
 +
django-guardian==2.4.0
 +
django-haystack==3.1.1
 +
django-imagekit==4.1.0
 +
django-invitations==1.9.3
 +
django-ipware==4.0.0
 +
django-js-asset==1.2.2
 +
django-jsonfield-backport==1.0.2
 +
django-leaflet==0.27.1
 +
django-markdownify==0.9.1
 +
django-model-utils==0.0.0
 +
django-modeltranslation==0.17.5
 +
django-mptt==0.13.2
 +
django-polymorphic==3.1.0
 +
django-recaptcha==2.0.6
 +
django-storages==1.12.3
 +
django-taggit==2.1.0
 +
django-tastypie==0.14.4
 +
django-timezone-field==5.0
 +
django-tinymce==3.3.0
 +
django-treebeard==4.5.1
 +
django-user-agents==0.4.0
 +
django-uuid-upload-path==1.0.0
 +
django-widget-tweaks==1.4.8
 +
djangorestframework-gis==0.18
 +
djangorestframework-guardian==0.3.0
 +
djangorestframework==3.12.4
 +
dnspython==2.1.0
 +
docker==5.0.3
 +
drf-extensions==0.7.1
 +
drf-spectacular==0.21.1
 +
drf-writable-nested==0.6.3
 +
dynamic-rest==2.0.0
 +
elasticsearch==7.16.2
 +
entrypoints==0.4
 +
ephem==4.1.3
 +
factory-boy==2.11.1
 +
flake8==4.0.1
 +
flaky==3.7.0
 +
folium==0.0.0
 +
fonttools==4.29.1
 +
frozenlist==1.2.0
 +
fs==2.4.12
 +
fsspec==2022.1.0
 +
future==0.18.2
 +
galternatives==1.0.8
 +
geographiclib==1.52
 +
geoip2==2.9.0
 +
geojson==2.5.0
 +
geolinks==0.2.0
 +
geonode-announcements==2.0.2
 +
geonode-avatar==5.0.7
 +
geonode-dialogos==3.0.1
 +
geonode-django-activity-stream==0.10.0
 +
geonode-oauth-toolkit==2.2.2
 +
geonode-user-messages==2.0.2
 +
geopandas==0.10.2
 +
geoserver-restconfig==2.0.7
 +
gisdata==0.5.4
 +
glob2==0.5
 +
gn-arcrest==10.5.4
 +
gn-gsimporter==2.0.1
 +
gps==3.22
 +
greenlet==1.1.2
 +
gunicorn==20.1.0
 +
hiredis==1.0.1
 +
html5lib==1.1
 +
httplib2==0.20.2
 +
humanize==0.0.0
 +
hyperlink==21.0.0
 +
idna==3.3
 +
importlib-metadata==4.6.4
 +
incremental==21.3.0
 +
inflection==0.3.1
 +
iniconfig==1.1.1
 +
invoke==1.4.1
 +
iotop==0.6
 +
ipykernel==6.7.0
 +
ipyleaflet==0.17.0
 +
ipython==7.31.1
 +
ipython_genutils==0.2.0
 +
ipywidgets==6.0.0
 +
iso8601==1.0.2
 +
isodate==0.6.1
 +
itsdangerous==2.1.0
 +
jdcal==1.0
 +
jedi==0.18.0
 +
jeepney==0.7.1
 +
jmespath==0.10.0
 +
joblib==0.17.0
 +
jsonfield==3.1.0
 +
jsonref==0.1
 +
jsonschema==3.2.0
 +
jupyter-client==7.1.2
 +
jupyter-core==4.9.1
 +
jupyter-packaging==0.11.1
 +
jupyterlab-pygments==0.1.2
 +
jwcrypto==1.0
 +
keyring==23.5.0
 +
kiwisolver==1.3.2
 +
kombu==5.2.3
 +
language-selector==0.1
 +
lark-parser==0.12.0
 +
launchpadlib==1.10.16
 +
lazr.restfulclient==0.14.4
 +
lazr.uri==1.0.6
 +
locket==0.2.1
 +
lubuntu-update-notifier==0.4.dev0
 +
lxml==4.8.0
 +
lz4==3.1.3+dfsg
 +
mapnik==3.0.23
 +
mappyfile==0.9.7
 +
mapscript==7.6.4
 +
matplotlib-inline==0.1.3
 +
matplotlib==3.5.1
 +
maxminddb==2.0.3
 +
mccabe==0.6.1
 +
mercantile==1.1.5
 +
mock==4.0.3
 +
more-itertools==8.10.0
 +
mpmath==0.0.0
 +
msal-extensions==1.0.0
 +
msal==1.17.0
 +
msgpack==1.0.3
 +
msrest==0.6.21
 +
msrestazure==0.6.4
 +
multidict==5.1.0
 +
munch==2.5.0
 +
nbclient==0.5.6
 +
nbconvert==6.4.0
 +
nbformat==5.1.3
 +
nest-asyncio==1.5.4
 +
netCDF4==1.5.8
 +
netifaces==0.11.0
 +
networkx==2.4
 +
notebook==6.4.8
 +
numpy==1.21.5
 +
oauthlib==3.2.0
 +
osmapi==3.0.0
 +
osmium==3.2.0
 +
packaging==21.3
 +
palettable==3.3.0
 +
pandas==1.3.5
 +
pandocfilters==1.5.0
 +
paramiko==2.9.3
 +
parse-type==0.5.6
 +
parse==1.19.0
 +
parso==0.8.1
 +
partd==1.2.0
 +
pbr==5.8.0
 +
pexpect==4.8.0
 +
pickleshare==0.7.5
 +
pilkit==2.0
 +
pinax-notifications==6.0.0
 +
pinax-ratings==4.0.0
 +
plotly==5.4.0
 +
pluggy==0.13.0
 +
ply==3.11
 +
portalocker==2.2.1
 +
prometheus-client==0.9.0
 +
prompt-toolkit==3.0.28
 +
psutil==5.9.0
 +
psycopg2==2.8.6
 +
ptyprocess==0.7.0
 +
py==1.10.0
 +
pyOpenSSL==21.0.0
 +
pyasn1-modules==0.2.1
 +
pyasn1==0.4.8
 +
pycairo==1.20.1
 +
pycodestyle==2.8.0
 +
pycountry==20.7.3
 +
pycql==0.0.8
 +
pycsw==2.6.1
 +
pycups==2.0.1
 +
pydantic==1.8.2
 +
pyflakes==2.4.0
 +
pygeoapi==0.12.0
 +
pylibmc==1.5.2
 +
pyows==0.2.6
 +
pyparsing==2.4.7
 +
pyproj==3.3.0
 +
pyrsistent==0.18.1
 +
pysal==2.1.0
 +
pyserial==3.5
 +
pyshp==2.2.0
 +
pytest-bdd==3.2.1
 +
pytest-django==3.5.1
 +
pytest==6.2.5
 +
python-apt==2.3.0+ubuntu2
 +
python-crontab==2.6.0
 +
python-dateutil==2.8.1
 +
python-debian===0.1.43ubuntu1
 +
python-etcd==0.4.5
 +
python-memcached==1.58
 +
python-mimeparse==1.6.0
 +
python-slugify==4.0.0
 +
python3-openid==3.2.0
 +
pytz==2022.1
 +
pywps==4.5.1
 +
pyxdg==0.27
 +
pyzmq==22.3.0
 +
rasterio==1.3.0
 +
redis==3.5.3
 +
requests-oauthlib==1.3.0
 +
requests-toolbelt==0.9.1
 +
requests==2.25.1
 +
s3transfer==0.5.0
 +
schema==0.6.7
 +
scikit-learn==0.23.2
 +
scipy==1.8.0
 +
seaborn==0.11.2
 +
selenium==4.0.0a1
 +
service-identity==18.1.0
 +
sherlock==0.3.2
 +
six==1.16.0
 +
smart-open==5.2.1
 +
snuggs==1.4.7
 +
sortedcontainers==2.1.0
 +
soupsieve==2.3.1
 +
sqlparse==0.4.2
 +
sympy==1.9
 +
systemd-python==234
 +
tblib==1.7.0
 +
tenacity==6.3.1
 +
terminado==0.13.1
 +
testpath==0.5.0
 +
threadpoolctl==3.1.0
 +
timeout-decorator==0.5.0
 +
tinydb==3.15.2
 +
toml==0.10.2
 +
tomlkit==0.9.2
 +
toolz==0.11.2
 +
tornado==6.1
 +
tqdm==4.57.0
 +
traitlets==5.1.1
 +
traittypes==0.2.1
 +
typing-extensions==3.10.0.2
 +
tzlocal==2.1
 +
ua-parser==0.10.0
 +
uamqp==1.5.1
 +
ubuntu-advantage-tools==27.7
 +
ubuntu-drivers-common==0.0.0
 +
ufoLib2==0.13.1
 +
ufw==0.36.1
 +
unattended-upgrades==0.1
 +
unicodecsv==0.14.1
 +
unicodedata2==14.0.0
 +
uritemplate==3.0.1
 +
urllib3==1.26.5
 +
usb-creator==0.3.7
 +
user-agents==2.2.0
 +
vine==5.0.0
 +
wadllib==1.3.6
 +
wcwidth==0.2.5
 +
webencodings==0.5.1
 +
websocket-client==1.2.3
 +
websockets==9.1
 +
widgetsnbextension==2.0.0
 +
wrapt==1.13.3
 +
wxPython==4.0.7
 +
xarray==0.16.1
 +
xdg==5
 +
xkit==0.0.0
 +
xmljson==0.1.9
 +
xmltodict==0.12.0
 +
xyzservices==2022.6.0
 +
yarl==1.7.2
 +
zict==2.0.0
 +
zipp==1.0.0
 +
zipstream==1.1.4
 +
zope.interface==5.4.0
 +
 
 +
</pre>
 +
 
 +
 
 +
 
 +
=== Focal Snapshot local ===
 +
<pre>
 +
                Sat 26 Feb 2022 07:10:31 PM PST
 +
 
 
$ ls .local/lib/python3.8/site-packages/ | grep info | sed -e 's/\.dist-info//'
 
$ ls .local/lib/python3.8/site-packages/ | grep info | sed -e 's/\.dist-info//'
 
aenum-3.1.8
 
aenum-3.1.8

Latest revision as of 17:41, 22 February 2023

Python Libraries Strategy

Fresh priority python3 for the Jupyter Notebooks of 2023

  • GeoPandas (and pySAL) : gpd 0.12direct link to GEOS 3.11, popular extension to Pandas data science; please recall that pySAL is merging into the GeoPandas project overall; GeoPandas+pySAL use matplotlib graphics base, prefers Folium for web maps (?)
  • Numpy & Pandas : np 1.21.5,pd 1.3.5 fundamental packages
  • PostGIS writing done with pyscopg2 2.9.2 (low-level) or geoalchemy2 0.10.2 (ORM based on sqlalchemy)
  • rasterio : modern geoTIFF handling; sophisticated library design; not easy to build from source 1.3.4
  • Shapely & Fiona : direct link to libGEOS/ogr from python; example Notebooks in place Shapely-1.8.5 Fiona-1.8.22
  • opendatacube core and Notebook client examples v1.8.7 -LINK- odc-geo a minimal Opendatacube setup (see below)
  • pySTAC spatial-temporal analysis standard; opendatacube collaborators v1.4 -LINK- updt v1.6
  • stactools is a collection of interfaces and utilities, TBD from odc soon -LINK-
  • Scikit-Learn : original machine-learning library by international research team led by French math; stands the tests of time; integrated with SciPy for Fortran, BLAS, LAPACK libs; numpy for data allocation; important primitives like PCA, SVM, and KNN are optimized in practical ways, and then benchmarked against pure Fortran or C++ for comparison 0.23.2
  • boto for AWS S3, other data store transfer; can use AWS credentials (or Hetzner, or..) modern version using pip installed locally
  • osmium libosmium2-dev, libosmium2-doc, python3-pyosmium, osmium-tool : a C++ framework for working with OSM data files; osmium library bindings for Python 3 ## osmium version 1.14.0; libosmium version 2.18.0; Supported PBF compression types: none zlib lz4; Copyright (C) 2013-2022 Jochen Topf <jochen@topf.org>
  • mapnik libmapnik3.1, python3-mapnik : mapnik rendering engine for OSM maps and others libmapnik.so.3.1.0
  • mapclassify : part of pySAL, versions must match with geopandas setup --LINK--
  • lightgbm : MSFT python, updt pip3 install --user --upgrade-strategy=only-if-needed lightgbm ; v3.3.2
  • inline maps vs slippy maps : cartopy, other matplotlib variants are inline static; notebook web maps need js libs e.g. ipyleaflet
  • EOmaps TODO evaluate here -LINK-
  • georasters TODO evaluate -LINK
  • Canada Met - Meteorological Service of Canada Open Data User Documentation -LINK-
  • LiDAR : this is slightly painful since modern commercial GIS is flooded with this tech, delivered by advanced Western companies seeking something novel to sell. Changes to the LASzip licensing have enabled inclusion, but as of today, no Debian maintainer in place, and bas wants PDAL out at this time. -EMAIL- -EMAIL2- *update* PDAL via dot-deb is included in ol15
  • DeepLearning "no" : deep learning has taken over in the modern AI/ML research, for funding and hype, but not on this disk. Please note that the ML libraries on osgeolive are completely useful, accurate and performant today.. no need for a constant and fast Internet connection, nor a compute infrastructure the size of a large naval ship, or special impenetrable models that can only be built by those machines. -LINK- Serious and in-demand applications like "facial recognition" are not included here. Many real applications to remote sensing are built using DeepLearning, but those are not available on this disk. Some benefits of not including DeepLearning : no dependencies on tricky GPU libs; not suffering from fast-updates of important ML core libraries; avoid black-box training models, and their disk space; avoid software patent problems in Western Nations.

"I feel that a common and well-known distro like this is an opportunity to build engineering cooperation, between teams that may not interact otherwise.. like the physical code sprints" -darkblueb in California

Example Notebooks in previous OSGeoLive -LINK-

Other Resources: Pangeo PYTHIA -LINK-

opendatacube "Cube in a Box" -LINK-

--

Jammy Notes

Using PPA ubuntugis-unstable & Ubuntu Universe dot-deb packages, the installation results look roughly like this: 

mjam@x10jam:~$ cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=22.04
DISTRIB_CODENAME=jammy
DISTRIB_DESCRIPTION="Ubuntu 22.04 LTS"

#--
 libgeos3.11.1   (libgeos3.11.1) 
 libgdal-dev     (3.6.2+dfsg-1~jammy0)
 postgresql-14-postgis-3     (3.3.2+dfsg-1~jammy0)
 PGSQL="140" POSTGIS="3.3.2" GEOS="3.11.1-CAPI-1.17.0" PROJ="9.1.1" LIBJSON="0.15" 
#--

System:
    python: 3.10.4 (main, Apr  2 2022, 09:04:19) [GCC 11.2.0]
executable: /usr/bin/python3
   machine: Linux-5.15.0-25-generic-x86_64-with-glibc2.35

Python dependencies:
          pip: 22.0.2
   setuptools: 59.6.0
      sklearn: 0.23.2
        numpy: 1.21.5
        scipy: 1.8.0
       Cython: None
       pandas: 1.3.5
   matplotlib: 3.5.1
       joblib: 0.17.0
threadpoolctl: 3.1.0

Built with OpenMP: True

Jupyter jammy Version

user@osgeolive:~$ jupyter --version
Selected Jupyter core packages...
IPython          : 7.31.1
ipykernel        : 6.7.0
ipywidgets       : 6.0.0
jupyter_client   : 7.1.2
jupyter_core     : 4.9.1
jupyter_server   : not installed
jupyterlab       : not installed
nbclient         : 0.5.6
nbconvert        : 6.4.0
nbformat         : 5.1.3
notebook         : 6.4.8
qtconsole        : not installed
traitlets        : 5.1.1

user@osgeolive:~$ jupyter --paths
config:
    /home/user/.jupyter
    /home/user/.local/etc/jupyter
    /usr/etc/jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/user/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/user/.local/share/jupyter/runtime




$ apt-get source python-dask-doc
Reading package lists... Done
Picking 'dask' as source package instead of 'python-dask-doc'
NOTICE: 'dask' packaging is maintained in the 'Git' version control system at:
https://salsa.debian.org/python-team/packages/dask.git
Please use:
git clone https://salsa.debian.org/python-team/packages/dask.git
to retrieve the latest (possibly unreleased) updates to the package.
Need to get 4,321 kB of source archives.
Get:1 http://mirrors.wikimedia.org/ubuntu jammy/universe dask 2022.01.0+dfsg-1ubuntu1 (dsc) ...            
Fetched 4,321 kB in 18s (236 kB/s)                                                                                  
dpkg-source: info: extracting dask in dask-2022.01.0+dfsg
dpkg-source: info: unpacking dask_2022.01.0+dfsg.orig.tar.xz
dpkg-source: info: unpacking dask_2022.01.0+dfsg-1ubuntu1.debian.tar.xz
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying pytest-futurewarning.patch
dpkg-source: info: applying remove-bokeh-embeds.patch
dpkg-source: info: applying use-debian-version.patch
dpkg-source: info: applying reproducible-version.patch
dpkg-source: info: applying Use-the-packaged-MathJax.patch
dpkg-source: info: applying use-local-intersphinx.patch
dpkg-source: info: applying use-local-reference-yaml.patch
dpkg-source: info: applying js-yaml.patch
dpkg-source: info: applying use-youtube-nocookie.patch
dpkg-source: info: applying reproducible-config-autofunction.patch
dpkg-source: info: applying 32bit-comatibility.patch
dpkg-source: info: applying scipy1.8.patch

Jupyter Notebooks

Notes: the demo material from Actinia/GRASS, opendatacube and others use an AWS connection. We want to demo without requiring a connection. There was an idea that a video could be a local file with the demo. Enock said he can edit video, and that some High School Summer of Code students submit their project this way, but the quality is not professional. Angelos suggested that any demo in a video should also be available to actually run, if the network is working; also the GRASS Notebooks download data, and then use that as the demo, instead of doing work only in the clouds. Brian said we should move to beta1 with ordinary Jupyter and then make changes to the details of the Notebooks and python stack.


opendatacube

The Open Data Cube (ODC) is an Open-Source Geospatial Data Management and Analysis Software project that helps you harness the power of Earth observation data. At its core, the ODC is a set of Python libraries and PostgreSQL database that helps you work with geospatial raster data.

Core features:

  • Catalogue large amounts of Earth observation data
  • Use a Python based API for query and data access
  • Perform exploratory data analysis using popular Python libraries like Xarray, Pandas, and Matplotlib

Details

Website: https://www.opendatacube.org/

Licence: Apache 2.0

Software Version: 1.8.7

Supported Platforms: Linux, Mac, Windows

API Interfaces: Python

25aug22 

FOSS4G   Pete Gadomski	pete.gadomski at Gmail 

        pystac  v1.6.1    r/w static collections, catalogs, objects
 pystac-client  v0.4.0    r/w stac api
     stactools  v0.4.1    create,modify utils
      odc-stac  v0.3.1    load stac assets into xarray

pystac version ref: https://trac.osgeo.org/osgeolive/ticket/2393



31july2022 

~$ DATACUBE_CONFIG_PATH=~/.config/datacube/datacube.conf datacube -v system init
2022-08-01 04:26:09,231 8491 datacube INFO Running datacube command: /usr/bin/datacube -v system init
Initialising database...
2022-08-01 04:26:10,032 8491 datacube.drivers.postgres._core INFO Ensuring user roles.
2022-08-01 04:26:10,044 8491 datacube.drivers.postgres._core INFO Creating schema.
2022-08-01 04:26:10,045 8491 datacube.drivers.postgres._core INFO Creating tables.
2022-08-01 04:26:10,093 8491 datacube.drivers.postgres._core INFO Creating triggers.
2022-08-01 04:26:10,100 8491 datacube.drivers.postgres._core INFO Creating added column.
2022-08-01 04:26:10,104 8491 datacube.drivers.postgres._core INFO Adding role grants.
2022-08-01 04:26:10,106 8491 datacube.index.postgres.index INFO Adding default metadata types.
Created.
Checking indexes/views.
2022-08-01 04:26:10,295 8491 datacube.drivers.postgres._api INFO Checking dynamic views/indexes. (rebuild views=True, indexes=False)
Done.

26july2022 

$ datacube product add /path/to/landsat-clip.yml

---  landsat-clip.yml
name: clip_landsat
description: example ortho imagery
metadata_type: eo3

license: CC-BY-4.0

metadata:
  product:
    name: clip_landsat

storage:
  crs: EPSG:32619
  resolution:
    longitude: 28.497418829384827
    latitude: -28.503958771796388

measurements:
  - name: imagery
    dtype: byte
    nodata: -32768.0
    units: "meter"

$ rio info data/landsat/clip10.tif 
{"bounds": [374141.9163735037, 4597362.958980484, 450657.48593040195, 4662494.504774039], "colorinterp": ["gray"], "count": 1, "crs": "EPSG:32619", "descriptions": [null], "driver": "GTiff", "dtype": "uint8", "height": 2285, "indexes": [1], "interleave": "band", "lnglat": [-70.054716274406, 41.81639512026538], "mask_flags": [["all_valid"]], "nodata": null, "res": [28.497418829384827, 28.503958771796388], "shape": [2285, 2685], "tiled": false, "transform": [28.497418829384827, 0.0, 374141.9163735037, 0.0, -28.503958771796388, 4662494.504774039, 0.0, 0.0, 1.0], "units": [null], "width": 2685}

# UUID of the dataset
id: f884df9b-4458-bbbb-aaaa-1a52a2db8a1a
$schema: 'https://schemas.opendatacube.org/dataset'

# Product name
product:
  name: clip_landsat_example0

# Native CRS, assumed to be the same across all bands
crs: "epsg:32619"

# Mapping name:str -> { shape:     Tuple[ny: int, nx: int]
#                       transform: Tuple[float x 9]}
# Captures image size, and geo-registration
grids:
    default:  # "default" grid must be present
       shape: [2285, 2685]
       transform: [28.497418829384827, 0.0, 374141.9163735037, 0.0, -28.503958771796388, 4662494.504774039, 0.0, 0.0, 1.0]

# Per band storage information and references into `grids`
# Bands using the "default" grid should not need to reference it
measurements:
   red:               # Band using "default" grid should omit `grid` key
     path: red.tif    # Path relative to the dataset location

# optional dataset location (useful for public datasets)
#location: https://landsatonaws.com/L8/099/072/LC08_L1GT_099072_20200523_20200523_01_RT/metadata.yaml

# Dataset properties, prefer STAC standard names here
# Timestamp is the only compulsory field here
properties:
  eo:platform: landsat-8
  eo:instrument: OLI_TIRS

  # When recording time range use dtr:{start,end}_datetime
  dtr:start_datetime: 2020-01-01T07:02:02.233Z
  dtr:end_datetime:   2020-01-01T07:03:04.397Z

  # ODC specific "extensions"
  odc:processing_datetime: 2020-02-02T08:10:00.000Z

  odc:file_format: GeoTIFF
  odc:region_code: "074071"   # provider specific unique identified for the same location
                              # for Landsat '{:03d}{:03d}'.format(path, row)

  dea:dataset_maturity: final # one of: final| interim| nrt (near real time)
  odc:product_family: ard     # can be useful for larger installations

# Lineage only references UUIDs of direct source datasets
# Mapping name:str -> [UUID]
lineage: {}  # set to empty object if no lineage is defined

25jul2022 

user@osgeolive:~$ pip install --user lark-parser
Collecting lark-parser
  Downloading lark_parser-0.12.0-py2.py3-none-any.whl (103 kB)
     ━---------------------------------━━━ 103.5/103.5 KB 352.8 kB/s eta 0:00:00
Installing collected packages: lark-parser
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.                      
mappyfile 0.9.7 requires jsonref==0.2, but you have jsonref 0.1 which is incompatible.            
Successfully installed lark-parser-0.12.0  

$ createdb datacube
$ psql datacube -c 'create extension postgis'
$ mkdir ~/.config/datacube   
$ cat << EOF >  ~/.config/datacube/datacube.conf
[datacube]
db_database: datacube

# A blank host will use a local socket. Specify a hostname (such as localhost) to use TCP.
db_hostname:

# Credentials are optional: you might have other Postgres authentication configured.
# The default username otherwise is the current user id.
db_username: user 
db_password: user
EOF

                     
user@osgeolive:~$ DATACUBE_CONFIG_PATH=~/.config/datacube/datacube.conf datacube -v system init
2022-07-26 04:35:51,760 4663 datacube INFO Running datacube command: /usr/bin/datacube -v system init
Initialising database...
2022-07-26 04:35:52,213 4663 datacube.drivers.postgres._core INFO Ensuring user roles.
2022-07-26 04:35:52,270 4663 datacube.drivers.postgres._core INFO Creating schema.
2022-07-26 04:35:52,271 4663 datacube.drivers.postgres._core INFO Creating tables.
2022-07-26 04:35:53,006 4663 datacube.drivers.postgres._core INFO Creating triggers.
2022-07-26 04:35:53,015 4663 datacube.drivers.postgres._core INFO Creating added column.
2022-07-26 04:35:53,055 4663 datacube.drivers.postgres._core INFO Adding role grants.
2022-07-26 04:35:53,066 4663 datacube.index.postgres.index INFO Adding default metadata types.
Created.
Checking indexes/views.
2022-07-26 04:35:53,376 4663 datacube.drivers.postgres._api INFO Checking dynamic views/indexes. (rebuild views=True, indexes=False)
Done.

https://github.com/opendatacube/datacube-dataset-config https://github.com/opendatacube/datacube-dataset-config/tree/main/products

https://mybinder.org/v2/gh/opendatacube/odc-stac/develop?urlpath=lab/workspaces/demo

https://notebooks.gesis.org/binder/jupyter/user/opendatacube-odc-stac-cnk43e3t/lab/workspaces/demo/tree/notebooks/stac-load-S2-ms.ipynb

https://datacube-core.readthedocs.io/en/latest/installation/indexing-data/step-guide.html

18June2022 - opendatacube notebooks have very current software under development. Finding the stable parts is a challenge. #osgeolive alpha1 python-geo environment is surprisingly current, and matches a lot of ODC team dev environments including Folium. BUT two important parts are different: odc-devs have more DASK components (multi-processing chains); they have extended rasterio parts that I have not seen here yet. Others not present on alpha1 are JupyterLab parts - we are not going to use those (I read in the reviews late last year JupyterLab still has quality problems). ipyleaflet is developed with JupyterLab.. I am not confident ipyleaflet will work.. maybe. The alpha1 dot-deb packaging simplifies setup, MUCH simpler than their dev setup, and we have no DeepLearning ML GPU libs here.

Minimal datacube install from dot-deb 

   $apt install  python3-xarray  python3-cfgrib  python3-datacube

   Packages that will be installed: python3-cachetools  python3-cloudpickle  python3-dask python3-datacube  python3-distributed  python3-fsspec python3-heapdict  python3-locket  python3-msgpack python3-partd  python3-psutil  python3-sortedcontainers python3-tblib  python3-toolz python3-tornado  python3-zict

Jupyter R Install

dbb@x10jam:~$ sudo apt install r-cran-sf
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done

The following additional packages will be installed:
  bzip2-doc gfortran gfortran-11 libblas-dev libbz2-dev libc-ares2 libgfortran-11-dev libjs-d3 libjs-es5-shim
  libjs-highlight.js libjs-jquery-datatables libjs-jquery-datatables-extensions libjs-jquery-selectize.js libjs-json
  libjs-microplugin.js libjs-modernizr libjs-prettify libjs-sifter.js libjs-twitter-bootstrap-datepicker liblapack-dev
  libnlopt0 libnode72 libreadline-dev libudunits2-0 libudunits2-data littler node-highlight.js node-html5shiv
  node-normalize.css nodejs nodejs-doc r-base-core r-base-dev r-cran-abind r-cran-acepack r-cran-askpass r-cran-assertthat
  r-cran-backports r-cran-base64enc r-cran-bench r-cran-bh r-cran-bit r-cran-bit64 r-cran-bitops r-cran-blob r-cran-boot
  r-cran-brio r-cran-broom r-cran-cachem r-cran-callr r-cran-caret r-cran-catools r-cran-cba r-cran-checkmate r-cran-chron
  r-cran-class r-cran-classint r-cran-cli r-cran-cliapp r-cran-cluster r-cran-codetools r-cran-colorspace r-cran-commonmark
  r-cran-conquer r-cran-covr r-cran-cpp11 r-cran-crayon r-cran-crosstalk r-cran-curl r-cran-cvst r-cran-data.table r-cran-dbi
  r-cran-dbitest r-cran-dbplyr r-cran-ddalpha r-cran-decor r-cran-deldir r-cran-deoptimr r-cran-desc r-cran-diffobj
  r-cran-digest r-cran-dimred r-cran-domc r-cran-dplyr r-cran-drr r-cran-dt r-cran-dygraphs r-cran-e1071 r-cran-earth
  r-cran-ellipsis r-cran-evaluate r-cran-fansi r-cran-farver r-cran-fastica r-cran-fastmap r-cran-fastmatch r-cran-filehash
  r-cran-fnn r-cran-fontbitstreamvera r-cran-fontliberation r-cran-fontquiver r-cran-foreach r-cran-foreign r-cran-formula
  r-cran-fs r-cran-fts r-cran-furrr r-cran-future r-cran-future.apply r-cran-gdata r-cran-generics r-cran-geometry
  r-cran-geosphere r-cran-ggplot2 r-cran-gh r-cran-gitcreds r-cran-globals r-cran-glue r-cran-goftest r-cran-gower
  r-cran-gplots r-cran-gridextra r-cran-gtable r-cran-gtools r-cran-hexbin r-cran-highr r-cran-hmisc r-cran-hms
  r-cran-htmltable r-cran-htmltools r-cran-htmlwidgets r-cran-httpuv r-cran-httr r-cran-igraph r-cran-ini r-cran-interp
  r-cran-ipred r-cran-isoband r-cran-iterators r-cran-jpeg r-cran-jsonlite r-cran-kernlab r-cran-kernsmooth r-cran-knitr
  r-cran-ks r-cran-labeling r-cran-later r-cran-lattice r-cran-latticeextra r-cran-lava r-cran-lazyeval r-cran-lifecycle
  r-cran-linprog r-cran-listenv r-cran-littler r-cran-lme4 r-cran-lobstr r-cran-logcondens r-cran-lpsolve r-cran-lubridate
  r-cran-lwgeom r-cran-magic r-cran-magrittr r-cran-mapproj r-cran-maps r-cran-maptools r-cran-markdown r-cran-mass
  r-cran-matrix r-cran-matrixmodels r-cran-matrixstats r-cran-mclust r-cran-mda r-cran-memoise r-cran-mets r-cran-mgcv
  r-cran-mime r-cran-minqa r-cran-misc3d r-cran-mlbench r-cran-mlmetrics r-cran-mockery r-cran-mockr r-cran-modeldata
  r-cran-modelmetrics r-cran-multcomp r-cran-multicool r-cran-munsell r-cran-mvtnorm r-cran-nlme r-cran-nloptr r-cran-nnet
  r-cran-numderiv r-cran-openssl r-cran-palmerpenguins r-cran-parallelly r-cran-pillar r-cran-pkgbuild r-cran-pkgconfig
  r-cran-pkgkitten r-cran-pkgload r-cran-plogr r-cran-plot3d r-cran-plotmo r-cran-plotrix r-cran-pls r-cran-plyr r-cran-png
  r-cran-polyclip r-cran-pracma r-cran-praise r-cran-prettycode r-cran-prettyunits r-cran-proc r-cran-processx r-cran-prodlim
  r-cran-profmem r-cran-progress r-cran-progressr r-cran-promises r-cran-proxy r-cran-ps r-cran-purrr r-cran-quadprog
  r-cran-quantmod r-cran-quantreg r-cran-r6 r-cran-ragg r-cran-rann r-cran-rappdirs r-cran-raster r-cran-rcolorbrewer
  r-cran-rcpp r-cran-rcpparmadillo r-cran-rcppeigen r-cran-rcppprogress r-cran-rcpproll r-cran-recipes r-cran-rematch2
  r-cran-reshape2 r-cran-rex r-cran-rgdal r-cran-rgl r-cran-rlang r-cran-rmarkdown r-cran-rmysql r-cran-robustbase r-cran-rocr
  r-cran-rpart r-cran-rpostgresql r-cran-rprojroot r-cran-rsample r-cran-rspectra r-cran-rsqlite r-cran-rstudioapi r-cran-runit
  r-cran-s2 r-cran-sandwich r-cran-sass r-cran-scales r-cran-selectr r-cran-setrng r-cran-sfsmisc r-cran-shiny r-cran-slider
  r-cran-sodium r-cran-sourcetools r-cran-sp r-cran-sparsem r-cran-spatial r-cran-spatstat r-cran-spatstat.core
  r-cran-spatstat.data r-cran-spatstat.geom r-cran-spatstat.linnet r-cran-spatstat.sparse r-cran-spatstat.utils r-cran-squarem
  r-cran-statmod r-cran-stringi r-cran-stringr r-cran-survival r-cran-svglite r-cran-sys r-cran-systemfonts
  r-cran-teachingdemos r-cran-tensor r-cran-terra r-cran-testit r-cran-testthat r-cran-textshaping r-cran-th.data r-cran-tibble
  r-cran-tidyr r-cran-tidyselect r-cran-tikzdevice r-cran-timedate r-cran-timereg r-cran-timeseries r-cran-tinytex
  r-cran-tseries r-cran-ttr r-cran-units r-cran-utf8 r-cran-vctrs r-cran-vdiffr r-cran-viridis r-cran-viridislite r-cran-waldo
  r-cran-warp r-cran-webutils r-cran-withr r-cran-wk r-cran-wkutils r-cran-xfun r-cran-xml r-cran-xml2 r-cran-xtable r-cran-xts
  r-cran-yaml r-cran-zoo r-doc-html r-recommended ttf-bitstream-vera


Get:1 http://mirrors.wikimedia.org/ubuntu jammy/main amd64 bzip2-doc all 1.0.8-5build1 [500 kB]

...                    
Get:325 http://mirrors.wikimedia.org/ubuntu jammy/universe amd64 r-recommended all 4.1.2-1ubuntu2 [2,624 B]   
                  
Fetched 299 MB in 12min 45s (391 kB/s)                                                                                          
Extracting templates from packages: 100%
Selecting previously unselected package bzip2-doc.
(Reading database ... 438665 files and directories currently installed.)

Setting up libjs-json (0~20210531+~1.0.8-1) ...
Setting up r-cran-bh (1.74.0-2) ...
Setting up bzip2-doc (1.0.8-5build1) ...
Setting up ttf-bitstream-vera (1.10-8.2) ...
Setting up r-doc-html (4.1.2-1ubuntu2) ...
Setting up libgfortran-11-dev:amd64 (11.2.0-19ubuntu1) ...
Setting up libjs-sifter.js (0.6.0+dfsg-2) ...
Setting up libjs-twitter-bootstrap-datepicker (1.3.1+dfsg1-4.1) ...
Setting up node-html5shiv (3.7.3+dfsg-4) ...
Setting up libnlopt0:amd64 (2.7.1-3build1) ...
Setting up libjs-microplugin.js (0.0.3+dfsg-1.1) ...
Setting up libc-ares2:amd64 (1.18.1-1build1) ...
Setting up libreadline-dev:amd64 (8.1.2-1) ...
Setting up libjs-modernizr (2.6.2+ds1-4) ...
Setting up libnode72:amd64 (12.22.9~dfsg-1ubuntu3) ...
Setting up libjs-jquery-datatables (1.11.4+dfsg-1) ...
Setting up libjs-es5-shim (4.6.4-1) ...
Setting up r-base-core (4.1.2-1ubuntu2) ...

Creating config file /etc/R/Renviron with new version

Note: https://isciences.gitlab.io/exactextractr/

pangeo test install

https://github.com/pangeo-data/pangeo-tutorial-gallery 

$ pip install --user geocube
Collecting geocube
  Downloading geocube-0.3.2-py3-none-any.whl (20 kB)
Requirement already satisfied: appdirs in /usr/lib/python3/dist-packages (from geocube) (1.4.4)
Requirement already satisfied: pyproj>=2 in /usr/lib/python3/dist-packages (from geocube) (3.3.0)
Requirement already satisfied: click>=6.0 in /usr/lib/python3/dist-packages (from geocube) (8.0.3)
Collecting rioxarray>=0.4
  Downloading rioxarray-0.11.1.tar.gz (49 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 49.7/49.7 KB 283.0 kB/s eta 0:00:00
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done
Collecting xarray>=0.17
  Downloading xarray-2022.3.0-py3-none-any.whl (870 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 870.9/870.9 KB 388.3 kB/s eta 0:00:00
Requirement already satisfied: numpy>=1.20 in /usr/lib/python3/dist-packages (from geocube) (1.21.5)
Requirement already satisfied: scipy in /usr/lib/python3/dist-packages (from geocube) (1.8.0)
Requirement already satisfied: geopandas>=0.7 in /usr/lib/python3/dist-packages (from geocube) (0.10.2)
Collecting odc-geo
  Downloading odc_geo-0.2.1-py3-none-any.whl (102 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 102.8/102.8 KB 279.6 kB/s eta 0:00:00
Requirement already satisfied: rasterio in /usr/lib/python3/dist-packages (from geocube) (1.2.10)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from rioxarray>=0.4->geocube) (21.3)
Requirement already satisfied: pandas>=1.1 in /usr/lib/python3/dist-packages (from xarray>=0.17->geocube) (1.3.5)
Requirement already satisfied: affine in /usr/lib/python3/dist-packages (from odc-geo->geocube) (2.3.0)
Requirement already satisfied: cachetools in /usr/lib/python3/dist-packages (from odc-geo->geocube) (5.0.0)
Requirement already satisfied: shapely in /usr/lib/python3/dist-packages (from odc-geo->geocube) (1.8.0)
Building wheels for collected packages: rioxarray
  Building wheel for rioxarray (pyproject.toml) ... done
  Created wheel for rioxarray: filename=rioxarray-0.11.1-py2.py3-none-any.whl size=50942 sha256=61bb0bfd8815420dc5129f7c7a14afff11796b6978c976a5b3001e54b16ac5e4
  Stored in directory: /home/dbb/.cache/pip/wheels/65/e9/a4/20811fc3ecaddf11c40d1e649d0d1bbe3cbbbd71559ca3cd9b
Successfully built rioxarray
Installing collected packages: xarray, odc-geo, rioxarray, geocube
  WARNING: The script geocube is installed in '/home/dbb/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed geocube-0.3.2 odc-geo-0.2.1 rioxarray-0.11.1 xarray-2022.3.0

env

   TODO pytest tests integration_tests in datacube-core




GDAL            3.4.3
numpy           1.21.5
scipy           1.8.0
dask            2022.1.0+dfsg
pandas          1.3.5
geopandas       0.10.2
ipython         7.31.1
matplotlib      3.5.1
pyproj          3.3.0
scikit-learn	0.23.2
seaborn         0.11.2
datacube        0.0.0

APScheduler	0.0.0
Automat	20.2.0
Babel	2.8.0
Brotli	1.0.9
Cartopy	0.0.0
Cheetah3	3.2.6
Deprecated	1.2.13
Django	2.2.24
EOxServer	1.1.2
Faker	0.9.3
Fiona	1.8.21
Flask	2.0.1
GeoAlchemy2	0.10.2
GeoNode	3.3.2.post2
HeapDict	1.0.1
Jinja2	3.0.3
Mako	1.1.3
MapProxy	1.15.1
Markdown	3.3.6
MarkupSafe	2.0.1
Nik4	1.7.0
OWSLib	0.25.0
Pillow	9.0.1
PyGObject	3.42.1
PyHamcrest	2.0.2
PyJWT	2.3.0
PyNaCl	1.5.0
PyOpenGL	3.1.5
PyQt5-sip	12.9.1
PyQt5	5.15.6
PyYAML	5.4.1
Pygments	2.11.2
QScintilla	2.11.6
Rtree	0.9.7
SQLAlchemy	1.4.31
SecretStorage	3.3.1
Send2Trash	1.8.1b0
Shapely	1.8.0
Twisted	22.1.0
Unidecode	1.3.3
Werkzeug	2.0.2
adal	1.2.2
affine	2.3.0
aiohttp	3.8.1
aioredis	1.3.1
aiosignal	1.2.0
amqp	5.0.9
anyjson	0.3.3
appdirs	1.4.4
apt-xapian-index	0.49
argon2-cffi	21.1.0
async-timeout	4.0.1
attrs	21.2.0
avro	1.11.0
azure-template	0.0.18b3
backcall	0.2.0
bcrypt	3.2.0
beautifulsoup4	4.10.0
billiard	3.6.4.0
bleach	4.1.0
blinker	1.4
boto3	1.20.34
botocore	1.23.34
branca	0.4.2
cachetools	5.0.0
celery	5.2.3
certifi	2020.6.20
cftime	1.5.2
chardet	4.0.0
charset-normalizer	2.0.6
click-didyoumean	0.0.3
click-plugins	1.1.1
click-repl	0.2.0
click	8.0.3
cligj	0.7.2
cloudpickle	2.0.0
colorama	0.4.4
command-not-found	0.3
constantly	15.1.0
coverage	6.2
croniter	1.0.15
cryptography	3.4.8
cupshelpers	1.0
cycler	0.11.0
dbus-python	1.2.18
decorator	4.4.2
defer	1.0.6
defusedxml	0.7.1
deprecation	2.0.7
descartes	1.1.0
distributed	2022.1.0+ds.1
distro-info	=1.1build1
distro	1.7.0
dj-database-url	0.5.0
dj-pagination	2.5.0
dnspython	2.1.0
docker	5.0.3
drf-extensions	0.7.1
drf-spectacular	0.21.1
drf-writable-nested	0.6.3
dynamic-rest	2.0.0
elasticsearch	7.16.2
entrypoints	0.4
ephem	4.1.3
factory-boy	2.11.1
flake8	4.0.1
flaky	3.7.0
folium	0.0.0
fonttools	4.29.1
frozenlist	1.2.0
fs	2.4.12
fsspec	2022.1.0
future	0.18.2
galternatives	1.0.8
geographiclib	1.52
geoip2	2.9.0
geojson	2.5.0
geolinks	0.2.0
geonode-announcements	2.0.2
geonode-avatar	5.0.7
geonode-dialogos	3.0.1
geonode-django-activity-stream	0.10.0
geonode-oauth-toolkit	2.2.2
geonode-user-messages	2.0.2
geoserver-restconfig	2.0.7
gisdata	0.5.4
glob2	0.5
gn-arcrest	10.5.4
gn-gsimporter	2.0.1
gps	3.22
greenlet	1.1.2
gunicorn	20.1.0
hiredis	1.0.1
html5lib	1.1
httplib2	0.20.2
humanize	0.0.0
hyperlink	21.0.0
idna	3.3
importlib-metadata	4.6.4
incremental	21.3.0
inflection	0.3.1
iniconfig	1.1.1
invoke	1.4.1
iotop	0.6
ipykernel	6.7.0
ipyleaflet	0.17.0
ipython_genutils	0.2.0
ipywidgets	6.0.0
iso8601	1.0.2
isodate	0.6.1
itsdangerous	2.1.0
jdcal	1.0
jedi	0.18.0
jeepney	0.7.1
jmespath	0.10.0
joblib	0.17.0
jsonfield	3.1.0
jsonref	0.1
jsonschema	3.2.0
jupyter-client	7.1.2
jupyter-core	4.9.1
jupyter-packaging	0.11.1
jupyterlab-pygments	0.1.2
jwcrypto	1.0
keyring	23.5.0
kiwisolver	1.3.2
kombu	5.2.3
language-selector	0.1
lark-parser	0.12.0
launchpadlib	1.10.16
lazr.restfulclient	0.14.4
lazr.uri	1.0.6
locket	0.2.1
lubuntu-update-notifier	0.4.dev0
lxml	4.8.0
lz4	3.1.3+dfsg
mapnik	3.0.23
mappyfile	0.9.7
mapscript	7.6.4
matplotlib-inline	0.1.3
maxminddb	2.0.3
mccabe	0.6.1
mercantile	1.1.5
mock	4.0.3
more-itertools	8.10.0
mpmath	0.0.0
msal-extensions	1.0.0
msal	1.17.0
msgpack	1.0.3
msrest	0.6.21
msrestazure	0.6.4
multidict	5.1.0
munch	2.5.0
nbclient	0.5.6
nbconvert	6.4.0
nbformat	5.1.3
nest-asyncio	1.5.4
netCDF4	1.5.8
netifaces	0.11.0
networkx	2.4
notebook	6.4.8
oauthlib	3.2.0
osmapi	3.0.0
osmium	3.2.0
packaging	21.3
palettable	3.3.0
pandocfilters	1.5.0
paramiko	2.9.3
parse-type	0.5.6
parse	1.19.0
parso	0.8.1
partd	1.2.0
pbr	5.8.0
pexpect	4.8.0
pickleshare	0.7.5
pilkit	2.0
pinax-notifications	6.0.0
pinax-ratings	4.0.0
plotly	5.4.0
pluggy	0.13.0
ply	3.11
portalocker	2.2.1
prometheus-client	0.9.0
prompt-toolkit	3.0.28
psutil	5.9.0
psycopg2	2.8.6
ptyprocess	0.7.0
py	1.10.0
pyOpenSSL	21.0.0
pyasn1-modules	0.2.1
pyasn1	0.4.8
pycairo	1.20.1
pycodestyle	2.8.0
pycountry	20.7.3
pycql	0.0.8
pycsw	2.6.1
pycups	2.0.1
pydantic	1.8.2
pyflakes	2.4.0
pygeoapi	0.12.0
pylibmc	1.5.2
pyows	0.2.6
pyparsing	2.4.7
pyrsistent	0.18.1
pysal	2.1.0
pyserial	3.5
pyshp	2.2.0
pytest-bdd	3.2.1
pytest-django	3.5.1
pytest	6.2.5
python-apt	2.3.0+ubuntu2
python-crontab	2.6.0
python-dateutil	2.8.1
python-debian	0.1.43ubuntu1
python-etcd	0.4.5
python-memcached	1.58
python-mimeparse	1.6.0
python-slugify	4.0.0
python3-openid	3.2.0
pytz	2022.1
pywps	4.5.1
pyxdg	0.27
pyzmq	22.3.0
rasterio	1.3.0
redis	3.5.3
requests-oauthlib	1.3.0
requests-toolbelt	0.9.1
requests	2.25.1
s3transfer	0.5.0
schema	0.6.7
selenium	4.0.0a1
service-identity	18.1.0
sherlock	0.3.2
six	1.16.0
smart-open	5.2.1
snuggs	1.4.7
sortedcontainers	2.1.0
soupsieve	2.3.1
sqlparse	0.4.2
sympy	1.9
systemd-python	234
tblib	1.7.0
tenacity	6.3.1
terminado	0.13.1
testpath	0.5.0
threadpoolctl	3.1.0
timeout-decorator	0.5.0
tinydb	3.15.2
toml	0.10.2
tomlkit	0.9.2
toolz	0.11.2
tornado	6.1
tqdm	4.57.0
traitlets	5.1.1
traittypes	0.2.1
typing-extensions	3.10.0.2
tzlocal	2.1
ua-parser	0.10.0
uamqp	1.5.1
ubuntu-advantage-tools	27.7
ubuntu-drivers-common	0.0.0
ufoLib2	0.13.1
ufw	0.36.1
unattended-upgrades	0.1
unicodecsv	0.14.1
unicodedata2	14.0.0
uritemplate	3.0.1
urllib3	1.26.5
usb-creator	0.3.7
user-agents	2.2.0
vine	5.0.0
wadllib	1.3.6
wcwidth	0.2.5
webencodings	0.5.1
websocket-client	1.2.3
websockets	9.1
widgetsnbextension	2.0.0
wrapt	1.13.3
wxPython	4.0.7
xarray	0.16.1
xdg	5
xkit	0.0.0
xmljson	0.1.9
xmltodict	0.12.0
xyzservices	2022.6.0
yarl	1.7.2
zict	2.0.0
zipp	1.0.0
zipstream	1.1.4
zope.interface	5.4.0

django-allauth	0.47.0
django-appconf	1.0.3
django-autocomplete-light	3.5.1
django-bootstrap3-datetimepicker-2	2.8.2
django-celery-beat	2.2.1
django-celery-results	2.0.0
django-cors-headers	3.11.0
django-downloadview	2.3.0
django-filter	2.4.0
django-floppyforms	1.9.0
django-forms-bootstrap	3.1.0
django-geonode-mapstore-client	3.3.2
django-grappelli	2.15.1
django-guardian	2.4.0
django-haystack	3.1.1
django-imagekit	4.1.0
django-invitations	1.9.3
django-ipware	4.0.0
django-js-asset	1.2.2
django-jsonfield-backport	1.0.2
django-leaflet	0.27.1
django-markdownify	0.9.1
django-model-utils	0.0.0
django-modeltranslation	0.17.5
django-mptt	0.13.2
django-polymorphic	3.1.0
django-recaptcha	2.0.6
django-storages	1.12.3
django-taggit	2.1.0
django-tastypie	0.14.4
django-timezone-field	5.0
django-tinymce	3.3.0
django-treebeard	4.5.1
django-user-agents	0.4.0
django-uuid-upload-path	1.0.0
django-widget-tweaks	1.4.8
djangorestframework-gis	0.18
djangorestframework-guardian	0.3.0
djangorestframework	3.12.4

azure-agrifood-farming	1.0.0b1
azure-ai-anomalydetector	3.0.0b3
azure-ai-formrecognizer	3.2.0b2
azure-ai-language-conversations	1.0.0b1
azure-ai-language-questionanswering	1.0.0
azure-ai-metricsadvisor	1.0.1
azure-ai-textanalytics	5.2.0b3
azure-ai-translation-document	1.0.0b6
azure-appconfiguration	1.3.0
azure-applicationinsights	0.1.0
azure-batch	11.0.0
azure-cognitiveservices-anomalydetector	0.3.0
azure-cognitiveservices-formrecognizer	0.1.1
azure-cognitiveservices-knowledge-qnamaker	0.3.0
azure-cognitiveservices-language-luis	0.7.0
azure-cognitiveservices-language-spellcheck	2.0.0
azure-cognitiveservices-language-textanalytics	0.2.0
azure-cognitiveservices-personalizer	0.1.0
azure-cognitiveservices-search-autosuggest	0.2.0
azure-cognitiveservices-search-customimagesearch	0.2.0
azure-cognitiveservices-search-customsearch	0.3.0
azure-cognitiveservices-search-entitysearch	2.0.0
azure-cognitiveservices-search-imagesearch	2.0.0
azure-cognitiveservices-search-newssearch	2.0.0
azure-cognitiveservices-search-videosearch	2.0.0
azure-cognitiveservices-search-visualsearch	0.2.0
azure-cognitiveservices-search-websearch	2.0.0
azure-cognitiveservices-vision-computervision	0.9.0
azure-cognitiveservices-vision-contentmoderator	1.0.0
azure-cognitiveservices-vision-customvision	3.1.0
azure-cognitiveservices-vision-face	0.5.0
azure-common	1.1.28
azure-communication-chat	1.1.0
azure-communication-identity	1.0.1
azure-communication-networktraversal	1.0.0b1
azure-communication-phonenumbers	1.0.1
azure-communication-sms	1.0.1
azure-confidentialledger	1.0.0b1
azure-containerregistry	1.0.0b7
azure-core	1.20.1
azure-data-tables	12.1.1
azure-digitaltwins-core	1.1.0
azure-eventgrid	4.7.0
azure-eventhub-checkpointstoreblob-aio	1.1.4
azure-eventhub-checkpointstoreblob	1.1.4
azure-eventhub-checkpointstoretable	1.0.0b1
azure-eventhub	5.6.2
azure-graphrbac	0.61.1
azure-identity	1.7.1
azure-iot-deviceupdate	1.0.0b1
azure-iot-modelsrepository	1.0.0b2
azure-keyvault-administration	4.1.0b2
azure-keyvault-certificates	4.4.0b2
azure-keyvault-keys	4.5.0b5
azure-keyvault-secrets	4.4.0b2
azure-keyvault	4.1.0
azure-loganalytics	0.1.0
azure-media-analytics-edge	1.0.0b1
azure-media-videoanalyzer-edge	1.0.0b3
azure-messaging-webpubsubservice	1.0.0b3
azure-mgmt-advisor	9.0.0
azure-mgmt-agfood	1.0.0b1
azure-mgmt-agrifood	1.0.0b1
azure-mgmt-alertsmanagement	1.0.0
azure-mgmt-apimanagement	2.1.0
azure-mgmt-appconfiguration	2.0.0
azure-mgmt-applicationinsights	1.0.0
azure-mgmt-appplatform	6.1.0
azure-mgmt-attestation	1.0.0
azure-mgmt-authorization	2.0.0
azure-mgmt-automanage	1.0.0b2
azure-mgmt-automation	1.1.0b1
azure-mgmt-avs	7.0.0b1
azure-mgmt-azureadb2c	1.0.0b1
azure-mgmt-azurearcdata	1.0.0
azure-mgmt-azurestack	1.0.0
azure-mgmt-azurestackhci	6.1.0b1
azure-mgmt-baremetalinfrastructure	1.0.0
azure-mgmt-batch	16.0.0
azure-mgmt-batchai	7.0.0b1
azure-mgmt-billing	6.0.0
azure-mgmt-botservice	1.0.0
azure-mgmt-cdn	11.0.0
azure-mgmt-changeanalysis	1.0.0
azure-mgmt-chaos	1.0.0b2
azure-mgmt-cognitiveservices	12.0.0
azure-mgmt-commerce	6.0.0
azure-mgmt-communication	1.0.0
azure-mgmt-compute	23.1.0
azure-mgmt-confidentialledger	1.0.0b1
azure-mgmt-confluent	2.0.0b1
azure-mgmt-consumption	8.0.0
azure-mgmt-containerinstance	9.1.0
azure-mgmt-containerregistry	8.2.0
azure-mgmt-containerservice	16.3.0
azure-mgmt-core	1.3.0
azure-mgmt-cosmosdb	7.0.0b2
azure-mgmt-costmanagement	3.0.0
azure-mgmt-customproviders	1.0.0
azure-mgmt-databox	1.0.0
azure-mgmt-databoxedge	1.0.0
azure-mgmt-databricks	1.1.0b1
azure-mgmt-datadog	2.0.0
azure-mgmt-datafactory	2.0.0
azure-mgmt-datalake-analytics	0.6.0
azure-mgmt-datalake-store	1.0.0
azure-mgmt-datamigration	10.0.0
azure-mgmt-dataprotection	1.0.0b1
azure-mgmt-datashare	1.0.0
azure-mgmt-deploymentmanager	1.0.0
azure-mgmt-deviceupdate	1.0.0b3
azure-mgmt-devspaces	1.0.0b1
azure-mgmt-devtestlabs	9.0.0
azure-mgmt-digitaltwins	6.0.0
azure-mgmt-dns	8.0.0
azure-mgmt-documentdb	0.1.3
azure-mgmt-edgegateway	0.1.0
azure-mgmt-edgeorder	1.0.0b1
azure-mgmt-elastic	1.0.0
azure-mgmt-eventgrid	9.0.0
azure-mgmt-eventhub	10.0.0
azure-mgmt-extendedlocation	1.0.0
azure-mgmt-fluidrelay	1.0.0b1
azure-mgmt-frontdoor	1.0.0
azure-mgmt-guestconfig	1.0.0b1
azure-mgmt-hanaonazure	1.0.0
azure-mgmt-hdinsight	9.0.0
azure-mgmt-healthbot	1.0.0b1
azure-mgmt-healthcareapis	1.1.0b1
azure-mgmt-hybridcompute	7.0.0
azure-mgmt-hybridkubernetes	1.1.0
azure-mgmt-hybridnetwork	1.0.0
azure-mgmt-imagebuilder	1.0.0b1
azure-mgmt-iotcentral	9.0.0b1
azure-mgmt-iothub	2.1.0
azure-mgmt-iothubprovisioningservices	1.0.0
azure-mgmt-keyvault	9.2.0
azure-mgmt-kubernetesconfiguration	1.0.0b1
azure-mgmt-kusto	2.1.0
azure-mgmt-labservices	1.0.0
azure-mgmt-loganalytics	11.0.0
azure-mgmt-logic	9.0.0
azure-mgmt-logz	1.0.0
azure-mgmt-machinelearningcompute	1.0.0b1
azure-mgmt-machinelearningservices	1.0.0
azure-mgmt-maintenance	2.0.0
azure-mgmt-managedservices	6.0.0
azure-mgmt-managementgroups	1.0.0
azure-mgmt-managementpartner	1.0.0
azure-mgmt-maps	2.0.0
azure-mgmt-marketplaceordering	1.1.0
azure-mgmt-media	8.0.0
azure-mgmt-mixedreality	1.0.0
azure-mgmt-monitor	2.0.0
azure-mgmt-msi	6.0.0b1
azure-mgmt-netapp	5.1.0
azure-mgmt-network	19.2.0
azure-mgmt-notificationhubs	7.0.0
azure-mgmt-operationsmanagement	1.0.0
azure-mgmt-peering	1.0.0
azure-mgmt-policyinsights	1.1.0b1
azure-mgmt-portal	1.0.0
azure-mgmt-powerbidedicated	1.0.0
azure-mgmt-powerbiembedded	2.0.0
azure-mgmt-privatedns	1.0.0
azure-mgmt-purview	1.0.0
azure-mgmt-quantum	1.0.0b2
azure-mgmt-quota	1.0.0b2
azure-mgmt-rdbms	10.0.0
azure-mgmt-recoveryservices	2.0.0
azure-mgmt-recoveryservicesbackup	3.0.0
azure-mgmt-recoveryservicessiterecovery	1.0.0b1
azure-mgmt-redhatopenshift	1.0.0
azure-mgmt-redis	13.0.0
azure-mgmt-redisenterprise	1.0.0
azure-mgmt-regionmove	1.0.0b1
azure-mgmt-relay	1.1.0
azure-mgmt-reservations	1.0.0
azure-mgmt-resource	20.0.0
azure-mgmt-resourcegraph	8.1.0b1
azure-mgmt-resourcehealth	1.0.0b1
azure-mgmt-resourcemover	1.1.0b2
azure-mgmt-scheduler	7.0.0b1
azure-mgmt-search	8.0.0
azure-mgmt-security	2.0.0b1
azure-mgmt-securityinsight	1.0.0b1
azure-mgmt-serialconsole	1.0.0
azure-mgmt-servermanager	2.0.0
azure-mgmt-servicebus	7.1.0
azure-mgmt-servicefabric	2.0.0
azure-mgmt-servicefabricmanagedclusters	1.0.0
azure-mgmt-servicelinker	1.0.0b1
azure-mgmt-signalr	1.0.0
azure-mgmt-sql	3.0.1
azure-mgmt-sqlvirtualmachine	1.0.0b1
azure-mgmt-storage	19.0.0
azure-mgmt-storagecache	1.1.0
azure-mgmt-storageimportexport	1.0.0b1
azure-mgmt-storagepool	1.0.0
azure-mgmt-storagesync	1.0.0
azure-mgmt-streamanalytics	1.0.0rc1
azure-mgmt-subscription	1.0.0
azure-mgmt-support	6.0.0
azure-mgmt-synapse	2.1.0b2
azure-mgmt-testbase	1.0.0b1
azure-mgmt-timeseriesinsights	1.0.0
azure-mgmt-trafficmanager	1.0.0b1
azure-mgmt-videoanalyzer	1.0.0b3
azure-mgmt-vmwarecloudsimple	1.0.0b1
azure-mgmt-web	5.0.0
azure-mgmt-webpubsub	1.0.0
azure-mgmt-workloadmonitor	1.0.0b2
azure-mixedreality-authentication	1.0.0b1
azure-monitor-opentelemetry-exporter	1.0.0b6
azure-monitor-query	1.0.1
azure-purview-administration	1.0.0b2
azure-purview-catalog	1.0.0b3
azure-purview-scanning	1.0.0b3
azure-schemaregistry-avroserializer	1.0.0b4
azure-schemaregistry	1.0.0
azure-search-documents	11.3.0b5
azure-security-attestation	1.0.1
azure-servicebus	7.4.0
azure-servicefabric	8.0.0.0
azure-servicemanagement-legacy	0.20.7
azure-storage-blob-changefeed	12.0.0b3
azure-storage-blob	12.9.0
azure-storage-file-datalake	12.5.0
azure-storage-file-share	12.6.0
azure-storage-queue	12.1.6
azure-synapse-accesscontrol	0.8.0
azure-synapse-artifacts	0.10.0
azure-synapse-managedprivateendpoints	0.5.0
azure-synapse-monitoring	0.3.0
azure-synapse-spark	0.8.0
azure-synapse	0.1.1
----------------------------- ----------------------------
Package                       Version

http://ci.terria.io/main/
https://datacube-core.readthedocs.io/en/latest/installation/database/setup.html
https://docs.xarray.dev/en/stable/user-guide/plotting.html#robust
https://github.com/opendatacube/datacube-core/releases/tag/1.8.7
https://github.com/opendatacube/odc-geo/issues
https://github.com/opendatacube/odc-geo/releases/tag/v0.2.0
https://github.com/opendatacube/odc-stac/releases/tag/v0.3.0
https://github.com/opendatacube/odc-stac/wiki/Generating-Rotated-Images-to-Save-Space
https://github.com/opendatacube/odc-tools/blob/develop/apps/dc_tools/odc/apps/dc_tools/fs_to_dc.py
https://maps.digitalearth.africa/
https://odc-geo.readthedocs.io/en/latest/_api/odc.geo.xr.ODCExtensionDs.html
https://odc-geo.readthedocs.io/en/latest/_api/odc.geo.xr.ODCExtension.html
https://odc-geo.readthedocs.io/en/latest/raster-tools.html#plotting-on-a-map
https://odc-stac-demo.netlify.app/
https://sandbox.digitalearth.africa/hub/login?next=%2Fhub%2F
https://www.usgs.gov/landsat-missions/landsat-collection-2-level-2-science-products

Alternately, you can configure the ODC connection to Postgres using environment variables:
DB_HOSTNAME
DB_USERNAME
DB_PASSWORD
DB_DATABASE


DevInstall via Paul Haesler on Slack #open-data-cube 

# install numpy, pyproj and rasterio from Ubuntu distro via apt
sudo apt-get install python3-numpy python3-rasterio python3-pyproj

# Create virtualenv inheriting numpy and rasterio from system site-packages.
cd ~/venvs
virtualenv -p `which python3` --system-site-packages odc

# Activate new virtualenv
source odc/bin/activate

# Compile and install cf-units from source:
cd ~/src
git clone git@github.com:SciTools/cf-units.git cf-units
cd cf-units
pip install -e .

# install datacube-core from source (from pypi should be fine after next release)
cd ~/src/datacube/core
pip install -e .[all]

odc-geo

$ pip install --user --upgrade-strategy only-if-needed odc-geo
Collecting odc-geo
  Downloading odc_geo-0.1.1-py3-none-any.whl (81 kB)
     |                              | 81 kB 254 kB/s 
Requirement already satisfied: affine in ./.local/lib/python3.8/site-packages (from odc-geo) (2.3.0)
Collecting cachetools
  Downloading cachetools-5.0.0-py3-none-any.whl (9.1 kB)
Requirement already satisfied: numpy in ./.local/lib/python3.8/site-packages (from odc-geo) (1.21.5)
Requirement already satisfied: pyproj in /usr/lib/python3/dist-packages (from odc-geo) (3.2.1)
Requirement already satisfied: shapely in /usr/lib/python3/dist-packages (from odc-geo) (1.8.0)
Installing collected packages: cachetools, odc-geo
Successfully installed cachetools-5.0.0 odc-geo-0.1.1

odc-stac

## from STAC-load-ms-S2.ipynb
import dask.distributed
import dask.utils
import numpy as np
import planetary_computer as pc
import xarray as xr
from IPython.display import display
from pystac_client import Client
from odc.stac import configure_rio, stac_load

## from STAC-load-e84-aws.ipynb
import dask.distributed
import folium
import folium.plugins
import geopandas as gpd
import shapely.geometry
from IPython.display import HTML, display
from pystac_client import Client
from odc.stac import configure_rio, stac_load

Focal Notes

Focal Setup Steps

APT install into the system OS python, major, stable parts first.. as a set .. ; 

  ## note gdal & postgis are installed at an earlier step via package dot-deb
  ## -------------------------------------------------------------------------
  591  sudo apt install python3-dev python3-pip
  405  sudo apt install ipython3 python3-scipy
            apt install python3-pandas  ## <== stale version currently
  408  sudo apt install r-cran-sf       ## match libgeos w/R language bindings
  429  sudo apt install python3-shapely python3-fiona
  614  sudo apt install --yes python3-geoalchemy2
  ##-- show the parts explicitly for Jupyter on Ubuntu via dot-deb ---------------
  670  sudo apt install --yes jupyter-core jupyter-client jupyter-console jupyter-nbformat jupyter-nbconvert  \
                              jupyter-notebook python3-ipykernel
  673  sudo apt install python-notebook-doc jupyter
  ##--
       sudo apt install python3-sklearn-lib  python3-pyosmium

next 

pip install --user  --upgrade-strategy=only-if-needed  py3-pkg
  # installs to `~/.local/lib/python3.8/` 
# 
  605  pip install --user   wheel
  835  pip install --user --upgrade  --upgrade-strategy=only-if-needed pandas 
  836  pip install --user --upgrade  --upgrade-strategy=only-if-needed scikit-image
  837  pip install --user --upgrade  --upgrade-strategy=only-if-needed imageio
  838  pip install --user --upgrade  --upgrade-strategy=only-if-needed numba
  630  pip install --user   folium
  839  pip install --user --upgrade  --upgrade-strategy=only-if-needed sentinelhub
  867  pip install --user --upgrade  --upgrade-strategy=only-if-needed openeo
  841  pip install --user --upgrade  --upgrade-strategy=only-if-needed eo-learn
  853  pip install --user --upgrade  --upgrade-strategy=only-if-needed ipyleaflet

  606  pip install --user   lightgbm  ## libs also have an R interface, optional engines

all [ rasterio, networkx, sklearn, boto, more...  ] into   ~/.local/lib/python3.8/  (below)

    note: https://pip.pypa.io/en/stable/cli/pip_install/#hash-checking-mode

next 

   $ mkdir -p ~/.jupyter/notebooks/ ;# default Jupyter notebooks & assets ; TODO will change.

this opinionated install will update/amend/evolve as installer code and more dot-debs are crafted later this year. 

$ ./do_start_jupyter.sh
#---------------------------
#!/bin/bash

jupyter notebook --no-browser --ip=192.168.1.131 --notebook-dir=.jupyter/notebooks


~/.bashrc

##-
PATH=$PATH:~/.local/bin




dbb@focalk:/etc$ cat lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"
#--
 libgeos3.10.1   (3.10.1-1~focal0) 
 libgdal-dev     (3.4.0+dfsg-1~focal0)
 postgresql-12-postgis-3     (3.2.0+dfsg-1~focal0)
 PGSQL="120" GEOS="3.10.1-CAPI-1.16.0" PROJ="8.2.0" LIBXML="2.9.10" LIBJSON="0.13.1"
#--


geoxarray

$ pip install --user rioxarray
Collecting rioxarray
  Downloading rioxarray-0.10.3.tar.gz (47 kB)
     |                              | 47 kB 272 kB/s 
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from rioxarray) (20.3)
Requirement already satisfied: xarray>=0.17 in ./.local/lib/python3.8/site-packages (from rioxarray) (0.21.1)
Requirement already satisfied: rasterio>=1.1.1 in ./.local/lib/python3.8/site-packages (from rioxarray) (1.2.10)
Requirement already satisfied: pyproj>=2.2 in /usr/lib/python3/dist-packages (from rioxarray) (3.2.1)
Requirement already satisfied: pandas>=1.1 in ./.local/lib/python3.8/site-packages (from xarray>=0.17->rioxarray) (1.4.1)
Requirement already satisfied: numpy>=1.18 in ./.local/lib/python3.8/site-packages (from xarray>=0.17->rioxarray) (1.21.5)
Requirement already satisfied: click-plugins in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (1.1.1)
Requirement already satisfied: cligj>=0.5 in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (0.5.0)
Requirement already satisfied: attrs in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (19.3.0)
Requirement already satisfied: certifi in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (2019.11.28)
Requirement already satisfied: click>=4.0 in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (7.0)
Requirement already satisfied: affine in ./.local/lib/python3.8/site-packages (from rasterio>=1.1.1->rioxarray) (2.3.0)
Requirement already satisfied: snuggs>=1.4.1 in ./.local/lib/python3.8/site-packages (from rasterio>=1.1.1->rioxarray) (1.4.7)
Requirement already satisfied: setuptools in /usr/lib/python3/dist-packages (from rasterio>=1.1.1->rioxarray) (45.2.0)
Requirement already satisfied: python-dateutil>=2.8.1 in ./.local/lib/python3.8/site-packages (from pandas>=1.1->xarray>=0.17->rioxarray) (2.8.2)
Requirement already satisfied: pytz>=2020.1 in ./.local/lib/python3.8/site-packages (from pandas>=1.1->xarray>=0.17->rioxarray) (2021.3)
Requirement already satisfied: pyparsing>=2.1.6 in /usr/lib/python3/dist-packages (from snuggs>=1.4.1->rasterio>=1.1.1->rioxarray) (2.4.6)
Requirement already satisfied: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.1->pandas>=1.1->xarray>=0.17->rioxarray) (1.14.0)
Building wheels for collected packages: rioxarray
  Building wheel for rioxarray (PEP 517) ... done
  Created wheel for rioxarray: filename=rioxarray-0.10.3-py3-none-any.whl size=55624 sha256=925199c830a54ca5e9b02e89060e822e27014db5cbe601fd6b513d106fe66687
  Stored in directory: /home/dbb/.cache/pip/wheels/42/71/40/298b7df47153454b8eb3ef8e5441dafa413e5a2d7ac1832b17
Successfully built rioxarray
Installing collected packages: rioxarray
Successfully installed rioxarray-0.10.3


OSM software stack

osm2pgsql and libosmium 


# jammy universe

    Thu May 26 07:55:03 PM PDT 2022

osm2pgsql   1.6.0+ds-1
libosmium2-dev  Version: 2.18.0-1

Sat 26 Feb 2022 07:10:31 PM PST

# ubuntugis-unstable  focal
osm2pgsql  1.3.0+ds-1~focal2

# git repo
osm2pgsql  1.6    lonvia committed on Jan 24 2022
osm2pgsql  1.5.1  Thu Jul 29 09:49:35 2021 +0200

# osm2pgsql cmake variables set
PROJ6_INCLUDE_DIR  /usr/include
PROJ_LIBRARY  /usr/lib/x86_64-linux-gnu/libproj.so
WITH_LUA  ON
EXTERNAL_LIBOSMIUM  FALSE
# from osm2pgsql/contrib/libosmium/
LIBOSMIUM_VERSION_STRING "2.17.0"

$ apt show libosmium2-dev  =>   2.17.2-1~focal0

## the result of apt installing Universe  python3-pyosmium  2.15.3-1build2
$ osmium --version  => osmium version 1.11.1, libosmium version 2.15.4

so, that is behind slightly from 2.17.x osmium headers

## rebuild osm2pgsql 1.6.0,   better !
$  osm2pgsql --version
2022-02-26 20:14:19  osm2pgsql version 1.6.0
Build: Debug
Compiled using the following library versions:
Libosmium 2.17.3
Proj [API 6] 8.2.0
Lua 5.3.3

$ sudo make install
  ...
[ 95%] Built target osm2pgsql_lib
[100%] Built target osm2pgsql
Install the project...
-- Install configuration: "Debug"
-- Installing: /usr/local/bin/osm2pgsql
-- Installing: /usr/local/bin/osm2pgsql-replication
-- Installing: /usr/local/share/man/man1/osm2pgsql-replication.1
-- Installing: /usr/local/share/osm2pgsql/default.style
-- Installing: /usr/local/share/osm2pgsql/empty.style
-- Installing: /usr/local/share/man/man1/osm2pgsql.1

    OSMnx  
      in tutorial   https://pygis.io/docs/d_access_osm.html
      code  https://pypi.org/project/osmnx/#files



## osm2pgsql FLEX layout post-processing via LUA

./flex-config/
  route-relations.lua   unitable.lua   geometries.lua
  data-types.lua   attributes.lua   compatible.lua
  simple.lua   places.lua   generic.lua   with-schema.lua

./tests/data/
  test_output_flex_multigeom_polygon.lua
  test_output_flex_way.lua
  test_output_flex_area_3857.lua
  test_output_flex_attr.lua
  test_output_flex_stage2.lua
  test_output_flex_nocluster.lua
  test_output_flex_multigeom_multipolygon_split.lua
  test_output_flex_multigeom.lua
  test_output_flex_multigeom_multipolygon.lua
  test_output_flex_nodes.lua
  test_output_flex_area_4326.lua
  test_output_flex_relation_combinations.lua
  test_output_flex_nogeom.lua
  test_output_flex_area.lua
  test_output_flex_lua_fail.lua
  test_output_flex_bbox.lua
  test_output_flex_types.lua
  test_output_flex_area_mix.lua
  test_output_flex_relation_changes.lua
  test_output_flex_schema.lua
  test_output_flex_multigeom_geometry_split.lua
  test_output_flex_stage2_alt.lua
  test_output_flex_uni.lua
  test_output_flex.lua
  test_output_flex_validgeom.lua
  test_output_flex_line.lua
  test_output_flex_invalid_geom.lua
  test_output_flex_multigeom_geometry.lua
  test_output_flex_relations.lua

## a full list of osmium verbs
osmium-add-locations-to-ways   osmium-export                  osmium-merge-changes
osmium-apply-changes           osmium-extract                 osmium-query-locations-index
osmium-cat                     osmium-file-formats            osmium-renumber
osmium-changeset-filter        osmium-fileinfo                osmium-show
osmium-check-refs              osmium-getid                   osmium-sort
osmium-create-locations-index  osmium-getparents              osmium-tags-filter
osmium-derive-changes          osmium-index-types             osmium-time-filter
osmium-diff                    osmium-merge

Focal Jupyter Overview

                                    Sat 19 Feb 2022 07:18:48 PM PST
$ jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : not installed
ipython          : 7.13.0
ipykernel        : 5.2.0
jupyter client   : 6.1.2
jupyter lab      : not installed
nbconvert        : 5.6.1
ipywidgets       : 6.0.0
nbformat         : 5.0.4
traitlets        : 4.3.3

##-------------------------------
Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
pandas version: 0.25.3   <== a weak spot in this setup
matplotlib version: 3.1.2
NumPy version: 1.17.4
SciPy version: 1.3.3
IPython version: 7.13.0
scikit-learn version: 1.0.2
##-----------------------------------------
dbb@focalk:~$ ipython3
Python 3.8.10 (default, Nov 26 2021, 20:14:08) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn                                                                                               
In [2]: from sklearn import show_versions                                                                            
In [3]: show_versions()                                                                                              

System:
    python: 3.8.10 (default, Nov 26 2021, 20:14:08)  [GCC 9.3.0]
executable: /usr/bin/python3
   machine: Linux-5.13.0-30-generic-x86_64-with-glibc2.29

Python dependencies:
          pip: 20.0.2
   setuptools: 45.2.0
      sklearn: 1.0.2
        numpy: 1.17.4
        scipy: 1.3.3
       Cython: 0.29.14
       pandas: 0.25.3      <== a weak spot in this setup
   matplotlib: 3.1.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
--
dbb@focalk:~$ jupyter --paths
config:
    /home/dbb/.jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/dbb/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/dbb/.local/share/jupyter/runtime

Focal APT Repos

dbb@focalk:/etc/apt$ cat sources.list
# Automatically generated by Calamares on 2022-01-02.
# Lines starting with "deb" are mandatory, while lines starting with "deb-src"
# are for more detailed package information.
...
-------------------------

dbb@focalk:/etc/apt$ cat sources.list.d/ubuntugis-ubuntu-ubuntugis-unstable-focal.list

deb http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main
deb-src http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main

eo-learn pip Fix-It Session

eolearn -- aprox. of python3 dependancies 

import 
:  pandas as pd
:  geopandas as gpd
:  shapely.  geometry, ops, wkt
:  rasterio. features, transform, warp
:  skimage.  feature, filters.rank, morphology, segmentation
:  boto3, cv2
:  fiona
:  graphviz
:  ------------------------------------------
:  abc, attr, base64
:  collections, concurrent.futures
:  dataclasses, datetime as dt
:  dateutil, dateutil.parser
:  fs, functools, gzip
:  importlib, inspect, itertools as it
:  json, logging, multiprocessing, os
:  pickle, ray, re, registration
:  tempfile, threading, time, traceback
:  unittest.mock as mock, uuid, warnings
: --------------------------

##-----------------------------------------------------------------
$ pip install --user --upgrade  --upgrade-strategy=only-if-needed    pandas
Collecting pandas
  Downloading pandas-1.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
     |                                | 11.7 MB 79 kB/s 
Collecting pytz>=2020.1
  Downloading pytz-2021.3-py2.py3-none-any.whl (503 kB)
     |                                | 503 kB 260 kB/s 
Collecting python-dateutil>=2.8.1
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     |                                | 247 kB 210 kB/s 
Collecting numpy>=1.18.5; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10"
  Downloading numpy-1.22.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
     |                                | 16.8 MB 11 kB/s 
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.1->pandas) (1.14.0)
ERROR: sentinelhub 3.4.4 has requirement pillow<=8.4.0,>=8.3.2, but you'll have pillow 7.0.0 which is incompatible.
ERROR: scikit-image 0.19.2 has requirement scipy>=1.4.1, but you'll have scipy 1.3.3 which is incompatible.
ERROR: numba 0.55.1 has requirement numpy<1.22,>=1.18, but you'll have numpy 1.22.2 which is incompatible.
ERROR: imageio 2.16.0 has requirement pillow>=8.3.2, but you'll have pillow 7.0.0 which is incompatible.
Installing collected packages: pytz, python-dateutil, numpy, pandas
Successfully installed numpy-1.22.2 pandas-1.4.1 python-dateutil-2.8.2 pytz-2021.3

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed     scikit-image
Requirement already up-to-date: scikit-image in ./.local/lib/python3.8/site-packages (0.19.2)
Requirement already satisfied, skipping upgrade: imageio>=2.4.1 in ./.local/lib/python3.8/site-packages (from scikit-image) (2.16.0)
Requirement already satisfied, skipping upgrade: packaging>=20.0 in /usr/lib/python3/dist-packages (from scikit-image) (20.3)
Collecting scipy>=1.4.1
  Downloading scipy-1.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.6 MB)
     |                                | 41.6 MB 20 kB/s 
Requirement already satisfied, skipping upgrade: pillow!=7.1.0,!=7.1.1,!=8.3.0,>=6.1.0 in /usr/lib/python3/dist-packages (from scikit-image) (7.0.0)
Requirement already satisfied, skipping upgrade: numpy>=1.17.0 in ./.local/lib/python3.8/site-packages (from scikit-image) (1.22.2)
Requirement already satisfied, skipping upgrade: PyWavelets>=1.1.1 in ./.local/lib/python3.8/site-packages (from scikit-image) (1.2.0)
Requirement already satisfied, skipping upgrade: networkx>=2.2 in ./.local/lib/python3.8/site-packages (from scikit-image) (2.6.3)
Requirement already satisfied, skipping upgrade: tifffile>=2019.7.26 in ./.local/lib/python3.8/site-packages (from scikit-image) (2022.2.9)
Installing collected packages: scipy
Successfully installed scipy-1.8.0

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      imageio
Requirement already up-to-date: imageio in ./.local/lib/python3.8/site-packages (2.16.0)
Requirement already satisfied, skipping upgrade: numpy>=1.20.0 in ./.local/lib/python3.8/site-packages (from imageio) (1.22.2)
Collecting pillow>=8.3.2
  Downloading Pillow-9.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
     |                                | 4.3 MB 99 kB/s 
ERROR: sentinelhub 3.4.4 has requirement pillow<=8.4.0,>=8.3.2, but you'll have pillow 9.0.1 which is incompatible.
Installing collected packages: pillow
Successfully installed pillow-9.0.1

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      numba
Requirement already up-to-date: numba in ./.local/lib/python3.8/site-packages (0.55.1)
Collecting numpy<1.22,>=1.18
  Downloading numpy-1.21.5-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
     |                                | 15.7 MB 361 kB/s 
Requirement already satisfied, skipping upgrade: setuptools in /usr/lib/python3/dist-packages (from numba) (45.2.0)
Requirement already satisfied, skipping upgrade: llvmlite<0.39,>=0.38.0rc1 in ./.local/lib/python3.8/site-packages (from numba) (0.38.0)
ERROR: sentinelhub 3.4.4 has requirement pillow<=8.4.0,>=8.3.2, but you'll have pillow 9.0.1 which is incompatible.
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.22.2
    Uninstalling numpy-1.22.2:
      Successfully uninstalled numpy-1.22.2
Successfully installed numpy-1.21.5

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed       sentinelhub
Requirement already up-to-date: sentinelhub in ./.local/lib/python3.8/site-packages (3.4.4)

Collecting pillow<=8.4.0,>=8.3.2
  Downloading Pillow-8.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
     |                                | 3.1 MB 43 kB/s 

Installing collected packages: pillow
  Attempting uninstall: pillow
    Found existing installation: Pillow 9.0.1
    Uninstalling Pillow-9.0.1:
      Successfully uninstalled Pillow-9.0.1
Successfully installed pillow-8.4.0

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      eo-learn
Requirement already up-to-date: eo-learn in ./.local/lib/python3.8/site-packages (1.0.0)

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      ipyleaflet
Collecting ipyleaflet
  Downloading ipyleaflet-0.15.0-py2.py3-none-any.whl (3.3 MB)
     |                                | 3.3 MB 364 kB/s 
Collecting traittypes<3,>=0.2.1
  Downloading traittypes-0.2.1-py2.py3-none-any.whl (8.6 kB)
Collecting xyzservices>=2021.8.1
  Downloading xyzservices-2022.2.0-py3-none-any.whl (35 kB)
Collecting ipywidgets<8,>=7.6.0
  Downloading ipywidgets-7.6.5-py2.py3-none-any.whl (121 kB)
     |                                | 121 kB 338 kB/s 
Requirement already satisfied, skipping upgrade: traitlets>=4.2.2 in /usr/lib/python3/dist-packages (from traittypes<3,>=0.2.1->ipyleaflet) (4.3.3)
Requirement already satisfied, skipping upgrade: ipykernel>=4.5.1 in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (5.2.0)
Requirement already satisfied, skipping upgrade: ipython>=4.0.0; python_version >= "3.3" in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (7.13.0)
Requirement already satisfied, skipping upgrade: ipython-genutils~=0.2.0 in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (0.2.0)
Collecting jupyterlab-widgets>=1.0.0; python_version >= "3.6"
  Downloading jupyterlab_widgets-1.0.2-py3-none-any.whl (243 kB)
     |                                | 243 kB 376 kB/s 
Collecting widgetsnbextension~=3.5.0
  Downloading widgetsnbextension-3.5.2-py2.py3-none-any.whl (1.6 MB)
     |                                | 1.6 MB 359 kB/s 
Requirement already satisfied, skipping upgrade: nbformat>=4.2.0 in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (5.0.4)
Requirement already satisfied, skipping upgrade: pexpect in /usr/lib/python3/dist-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets<8,>=7.6.0->ipyleaflet) (4.6.0)
Requirement already satisfied, skipping upgrade: notebook>=4.4.1 in /usr/lib/python3/dist-packages (from widgetsnbextension~=3.5.0->ipywidgets<8,>=7.6.0->ipyleaflet) (6.0.3)
Installing collected packages: traittypes, xyzservices, jupyterlab-widgets, widgetsnbextension, ipywidgets, ipyleaflet
Successfully installed ipyleaflet-0.15.0 ipywidgets-7.6.5 jupyterlab-widgets-1.0.2 traittypes-0.2.1 widgetsnbextension-3.5.2 xyzservices-2022.2.0

##jupyter nbextension enable --py --sys-prefix{?} ipyleaflet  # can be skipped for notebook 5.3 and above

R setup


--
R language example 

''Explainable Machine Learning for Land Cover Classification'' by Courage K.  2021

<pre>
# Load the libraries example code from aigeolabs.com
rm(list=ls())
library(sf)    # represents simple features as native R objects
library(rgdal) # interface to the “Geospatial Data Abstraction Library (GDAL)
library(raster) # provides important functions for importing and handling raster data
library(rasterVis) # provides functions to display raster objects
library(dplyr) # provides basic data transformation functions
library(ggplot2) # provides extension for visualizations
library(mapview) # provides functions to visualize geospatial data
library(randomForest) # implements Breiman’s random forest (RF) algorithm
library(kernlab) # implements support vector machine (SVM) algorithm
library(corrplot) # provides graphical display of a correlation matrix
library(iml)   # Keras-base tools for analyzing machine learning models
library(caret) #  machine learning helper

##--------------



OSGeoLive install_R.sh ver14

apt-get --assume-yes install r-base-core r-cran-sp r-cran-rpostgresql r-cran-udunits2 \
  r-cran-ggplot2 r-cran-sf r-cran-lwgeom r-cran-spdata r-cran-rgdal r-cran-maptools


osgeolive 10 patch

in 2016, after being alerted -LINK- , I used pip to upgrade ipywidgets on the OSGeo-Live 10.0 base system.

The following is a modified patch file showing the changes in Python 2.7: 


## Major components, version changed
-ipykernel==4.2.2
+ipykernel==4.5.2
...
-ipython==4.0.2
+ipython==5.1.0
-ipywidgets==5.1.5
+ipywidgets==5.2.2
-jupyter-client==4.1.1
+jupyter-client==4.4.0
 jupyter-console==4.1.0
-jupyter-core==4.0.6
+jupyter-core==4.2.1
-notebook==4.2.1
+notebook==4.3.0
-nbconvert==4.1.0
-nbformat==4.0.1
+nbconvert==5.0.0
+nbformat==4.2.0
+html5lib==0.9999999
-ptyprocess==0.5
+ptyprocess==0.5.1
-pexpect==4.0.1
-pickleshare==0.5
+pexpect==4.2.1
+pickleshare==0.7.4

-decorator==4.0.6
+decorator==4.0.10

##-- misc other additions --
+backports-abc==0.5
+backports.shutil-get-terminal-size==1.0.0
+bleach==1.5.0
+certifi==2016.9.26
+configparser==3.5.0
-enum34==1.1.2
+entrypoints==0.2.2
+enum34==1.1.6
-mistune==0.7.1
+mistune==0.7.3
+pandocfilters==1.4.1
+pathlib2==2.1.0
+prompt-toolkit==1.0.9
-Pygments==2.1
+Pygments==2.1.3
-pyzmq==15.2.0
+pyzmq==16.0.2
+testpath==0.3
-tornado==4.2.1
-traitlets==4.1.0
+tornado==4.4.2
+traitlets==4.3.1
+wcwidth==0.1.7
-widgetsnbextension==1.2.3
+widgetsnbextension==1.2.6

##-- misc other python libs (unchanged)
 anyjson==0.3.3
 Biggus==0.13.0
 Cartopy==0.14.2
 celery==3.1.20
 cf-units==1.0.0
 colorama==0.3.7
 cycler==0.9.0
 defer==1.0.6
 descartes==1.0.1
 filemagic==1.6
 Fiona==1.6.3
 Iris==1.9.2
 imposm==2.6.0
 imposm.parser==1.0.7
 ipyleaflet==0.2.0
 jsonschema==2.5.1
 meld==3.14.2
 ModestMaps==1.4.6
 netCDF4==1.2.4
 Nik4==1.6.0
 numpy==1.11.0
 osmapi==0.6.2
 OWSLib==0.10.3
 pandas==0.17.1
 psutil==3.4.2
 psycopg2==2.6.1
 pykml==0.1.1
 pywps==3.2.5
 rasterio==0.31.0
 TileStache==1.49.8

Python Packages All

jammy Snapshot python install

        osgeolive rc1 python3_all

APScheduler==0.0.0
Automat==20.2.0
Babel==2.8.0
Brotli==1.0.9
Cartopy==0.0.0
Cheetah3==3.2.6
Deprecated==1.2.13
Django==2.2.24
EOxServer==1.1.2
Faker==0.9.3
Fiona==1.8.21
Flask==2.0.1
GDAL==3.4.3
GeoAlchemy2==0.10.2
GeoNode==3.3.2.post2
HeapDict==1.0.1
Jinja2==3.0.3
Mako==1.1.3
MapProxy==1.15.1
Markdown==3.3.6
MarkupSafe==2.0.1
Nik4==1.7.0
OWSLib==0.25.0
Pillow==9.0.1
PyGObject==3.42.1
PyHamcrest==2.0.2
PyJWT==2.3.0
PyNaCl==1.5.0
PyOpenGL==3.1.5
PyQt5-sip==12.9.1
PyQt5==5.15.6
PyYAML==5.4.1
Pygments==2.11.2
QScintilla==2.11.6
Rtree==0.9.7
SQLAlchemy==1.4.31
SecretStorage==3.3.1
Send2Trash==1.8.1b0
Shapely==1.8.0
Twisted==22.1.0
Unidecode==1.3.3
Werkzeug==2.0.2
adal==1.2.2
affine==2.3.0
aiohttp==3.8.1
aioredis==1.3.1
aiosignal==1.2.0
amqp==5.0.9
anyjson==0.3.3
appdirs==1.4.4
apt-xapian-index==0.49
argon2-cffi==21.1.0
async-timeout==4.0.1
attrs==21.2.0
avro==1.11.0
azure-agrifood-farming==1.0.0b1
azure-ai-anomalydetector==3.0.0b3
azure-ai-formrecognizer==3.2.0b2
azure-ai-language-conversations==1.0.0b1
azure-ai-language-questionanswering==1.0.0
azure-ai-metricsadvisor==1.0.1
azure-ai-textanalytics==5.2.0b3
azure-ai-translation-document==1.0.0b6
azure-appconfiguration==1.3.0
azure-applicationinsights==0.1.0
azure-batch==11.0.0
azure-cognitiveservices-anomalydetector==0.3.0
azure-cognitiveservices-formrecognizer==0.1.1
azure-cognitiveservices-knowledge-qnamaker==0.3.0
azure-cognitiveservices-language-luis==0.7.0
azure-cognitiveservices-language-spellcheck==2.0.0
azure-cognitiveservices-language-textanalytics==0.2.0
azure-cognitiveservices-personalizer==0.1.0
azure-cognitiveservices-search-autosuggest==0.2.0
azure-cognitiveservices-search-customimagesearch==0.2.0
azure-cognitiveservices-search-customsearch==0.3.0
azure-cognitiveservices-search-entitysearch==2.0.0
azure-cognitiveservices-search-imagesearch==2.0.0
azure-cognitiveservices-search-newssearch==2.0.0
azure-cognitiveservices-search-videosearch==2.0.0
azure-cognitiveservices-search-visualsearch==0.2.0
azure-cognitiveservices-search-websearch==2.0.0
azure-cognitiveservices-vision-computervision==0.9.0
azure-cognitiveservices-vision-contentmoderator==1.0.0
azure-cognitiveservices-vision-customvision==3.1.0
azure-cognitiveservices-vision-face==0.5.0
azure-common==1.1.28
azure-communication-chat==1.1.0
azure-communication-identity==1.0.1
azure-communication-networktraversal==1.0.0b1
azure-communication-phonenumbers==1.0.1
azure-communication-sms==1.0.1
azure-confidentialledger==1.0.0b1
azure-containerregistry==1.0.0b7
azure-core==1.20.1
azure-data-tables==12.1.1
azure-digitaltwins-core==1.1.0
azure-eventgrid==4.7.0
azure-eventhub-checkpointstoreblob-aio==1.1.4
azure-eventhub-checkpointstoreblob==1.1.4
azure-eventhub-checkpointstoretable==1.0.0b1
azure-eventhub==5.6.2
azure-graphrbac==0.61.1
azure-identity==1.7.1
azure-iot-deviceupdate==1.0.0b1
azure-iot-modelsrepository==1.0.0b2
azure-keyvault-administration==4.1.0b2
azure-keyvault-certificates==4.4.0b2
azure-keyvault-keys==4.5.0b5
azure-keyvault-secrets==4.4.0b2
azure-keyvault==4.1.0
azure-loganalytics==0.1.0
azure-media-analytics-edge==1.0.0b1
azure-media-videoanalyzer-edge==1.0.0b3
azure-messaging-webpubsubservice==1.0.0b3
azure-mgmt-advisor==9.0.0
azure-mgmt-agfood==1.0.0b1
azure-mgmt-agrifood==1.0.0b1
azure-mgmt-alertsmanagement==1.0.0
azure-mgmt-apimanagement==2.1.0
azure-mgmt-appconfiguration==2.0.0
azure-mgmt-applicationinsights==1.0.0
azure-mgmt-appplatform==6.1.0
azure-mgmt-attestation==1.0.0
azure-mgmt-authorization==2.0.0
azure-mgmt-automanage==1.0.0b2
azure-mgmt-automation==1.1.0b1
azure-mgmt-avs==7.0.0b1
azure-mgmt-azureadb2c==1.0.0b1
azure-mgmt-azurearcdata==1.0.0
azure-mgmt-azurestack==1.0.0
azure-mgmt-azurestackhci==6.1.0b1
azure-mgmt-baremetalinfrastructure==1.0.0
azure-mgmt-batch==16.0.0
azure-mgmt-batchai==7.0.0b1
azure-mgmt-billing==6.0.0
azure-mgmt-botservice==1.0.0
azure-mgmt-cdn==11.0.0
azure-mgmt-changeanalysis==1.0.0
azure-mgmt-chaos==1.0.0b2
azure-mgmt-cognitiveservices==12.0.0
azure-mgmt-commerce==6.0.0
azure-mgmt-communication==1.0.0
azure-mgmt-compute==23.1.0
azure-mgmt-confidentialledger==1.0.0b1
azure-mgmt-confluent==2.0.0b1
azure-mgmt-consumption==8.0.0
azure-mgmt-containerinstance==9.1.0
azure-mgmt-containerregistry==8.2.0
azure-mgmt-containerservice==16.3.0
azure-mgmt-core==1.3.0
azure-mgmt-cosmosdb==7.0.0b2
azure-mgmt-costmanagement==3.0.0
azure-mgmt-customproviders==1.0.0
azure-mgmt-databox==1.0.0
azure-mgmt-databoxedge==1.0.0
azure-mgmt-databricks==1.1.0b1
azure-mgmt-datadog==2.0.0
azure-mgmt-datafactory==2.0.0
azure-mgmt-datalake-analytics==0.6.0
azure-mgmt-datalake-store==1.0.0
azure-mgmt-datamigration==10.0.0
azure-mgmt-dataprotection==1.0.0b1
azure-mgmt-datashare==1.0.0
azure-mgmt-deploymentmanager==1.0.0
azure-mgmt-deviceupdate==1.0.0b3
azure-mgmt-devspaces==1.0.0b1
azure-mgmt-devtestlabs==9.0.0
azure-mgmt-digitaltwins==6.0.0
azure-mgmt-dns==8.0.0
azure-mgmt-documentdb==0.1.3
azure-mgmt-edgegateway==0.1.0
azure-mgmt-edgeorder==1.0.0b1
azure-mgmt-elastic==1.0.0
azure-mgmt-eventgrid==9.0.0
azure-mgmt-eventhub==10.0.0
azure-mgmt-extendedlocation==1.0.0
azure-mgmt-fluidrelay==1.0.0b1
azure-mgmt-frontdoor==1.0.0
azure-mgmt-guestconfig==1.0.0b1
azure-mgmt-hanaonazure==1.0.0
azure-mgmt-hdinsight==9.0.0
azure-mgmt-healthbot==1.0.0b1
azure-mgmt-healthcareapis==1.1.0b1
azure-mgmt-hybridcompute==7.0.0
azure-mgmt-hybridkubernetes==1.1.0
azure-mgmt-hybridnetwork==1.0.0
azure-mgmt-imagebuilder==1.0.0b1
azure-mgmt-iotcentral==9.0.0b1
azure-mgmt-iothub==2.1.0
azure-mgmt-iothubprovisioningservices==1.0.0
azure-mgmt-keyvault==9.2.0
azure-mgmt-kubernetesconfiguration==1.0.0b1
azure-mgmt-kusto==2.1.0
azure-mgmt-labservices==1.0.0
azure-mgmt-loganalytics==11.0.0
azure-mgmt-logic==9.0.0
azure-mgmt-logz==1.0.0
azure-mgmt-machinelearningcompute==1.0.0b1
azure-mgmt-machinelearningservices==1.0.0
azure-mgmt-maintenance==2.0.0
azure-mgmt-managedservices==6.0.0
azure-mgmt-managementgroups==1.0.0
azure-mgmt-managementpartner==1.0.0
azure-mgmt-maps==2.0.0
azure-mgmt-marketplaceordering==1.1.0
azure-mgmt-media==8.0.0
azure-mgmt-mixedreality==1.0.0
azure-mgmt-monitor==2.0.0
azure-mgmt-msi==6.0.0b1
azure-mgmt-netapp==5.1.0
azure-mgmt-network==19.2.0
azure-mgmt-notificationhubs==7.0.0
azure-mgmt-operationsmanagement==1.0.0
azure-mgmt-peering==1.0.0
azure-mgmt-policyinsights==1.1.0b1
azure-mgmt-portal==1.0.0
azure-mgmt-powerbidedicated==1.0.0
azure-mgmt-powerbiembedded==2.0.0
azure-mgmt-privatedns==1.0.0
azure-mgmt-purview==1.0.0
azure-mgmt-quantum==1.0.0b2
azure-mgmt-quota==1.0.0b2
azure-mgmt-rdbms==10.0.0
azure-mgmt-recoveryservices==2.0.0
azure-mgmt-recoveryservicesbackup==3.0.0
azure-mgmt-recoveryservicessiterecovery==1.0.0b1
azure-mgmt-redhatopenshift==1.0.0
azure-mgmt-redis==13.0.0
azure-mgmt-redisenterprise==1.0.0
azure-mgmt-regionmove==1.0.0b1
azure-mgmt-relay==1.1.0
azure-mgmt-reservations==1.0.0
azure-mgmt-resource==20.0.0
azure-mgmt-resourcegraph==8.1.0b1
azure-mgmt-resourcehealth==1.0.0b1
azure-mgmt-resourcemover==1.1.0b2
azure-mgmt-scheduler==7.0.0b1
azure-mgmt-search==8.0.0
azure-mgmt-security==2.0.0b1
azure-mgmt-securityinsight==1.0.0b1
azure-mgmt-serialconsole==1.0.0
azure-mgmt-servermanager==2.0.0
azure-mgmt-servicebus==7.1.0
azure-mgmt-servicefabric==2.0.0
azure-mgmt-servicefabricmanagedclusters==1.0.0
azure-mgmt-servicelinker==1.0.0b1
azure-mgmt-signalr==1.0.0
azure-mgmt-sql==3.0.1
azure-mgmt-sqlvirtualmachine==1.0.0b1
azure-mgmt-storage==19.0.0
azure-mgmt-storagecache==1.1.0
azure-mgmt-storageimportexport==1.0.0b1
azure-mgmt-storagepool==1.0.0
azure-mgmt-storagesync==1.0.0
azure-mgmt-streamanalytics==1.0.0rc1
azure-mgmt-subscription==1.0.0
azure-mgmt-support==6.0.0
azure-mgmt-synapse==2.1.0b2
azure-mgmt-testbase==1.0.0b1
azure-mgmt-timeseriesinsights==1.0.0
azure-mgmt-trafficmanager==1.0.0b1
azure-mgmt-videoanalyzer==1.0.0b3
azure-mgmt-vmwarecloudsimple==1.0.0b1
azure-mgmt-web==5.0.0
azure-mgmt-webpubsub==1.0.0
azure-mgmt-workloadmonitor==1.0.0b2
azure-mixedreality-authentication==1.0.0b1
azure-monitor-opentelemetry-exporter==1.0.0b6
azure-monitor-query==1.0.1
azure-purview-administration==1.0.0b2
azure-purview-catalog==1.0.0b3
azure-purview-scanning==1.0.0b3
azure-schemaregistry-avroserializer==1.0.0b4
azure-schemaregistry==1.0.0
azure-search-documents==11.3.0b5
azure-security-attestation==1.0.1
azure-servicebus==7.4.0
azure-servicefabric==8.0.0.0
azure-servicemanagement-legacy==0.20.7
azure-storage-blob-changefeed==12.0.0b3
azure-storage-blob==12.9.0
azure-storage-file-datalake==12.5.0
azure-storage-file-share==12.6.0
azure-storage-queue==12.1.6
azure-synapse-accesscontrol==0.8.0
azure-synapse-artifacts==0.10.0
azure-synapse-managedprivateendpoints==0.5.0
azure-synapse-monitoring==0.3.0
azure-synapse-spark==0.8.0
azure-synapse==0.1.1
azure-template==0.0.18b3
backcall==0.2.0
bcrypt==3.2.0
beautifulsoup4==4.10.0
billiard==3.6.4.0
bleach==4.1.0
blinker==1.4
boto3==1.20.34
botocore==1.23.34
branca==0.4.2
cachetools==5.0.0
celery==5.2.3
certifi==2020.6.20
cftime==1.5.2
chardet==4.0.0
charset-normalizer==2.0.6
click-didyoumean==0.0.3
click-plugins==1.1.1
click-repl==0.2.0
click==8.0.3
cligj==0.7.2
cloudpickle==2.0.0
colorama==0.4.4
command-not-found==0.3
constantly==15.1.0
coverage==6.2
croniter==1.0.15
cryptography==3.4.8
cupshelpers==1.0
cycler==0.11.0
dask==2022.1.0+dfsg
datacube==0.0.0
dbus-python==1.2.18
decorator==4.4.2
defer==1.0.6
defusedxml==0.7.1
deprecation==2.0.7
descartes==1.1.0
distributed==2022.1.0+ds.1
distro-info===1.1build1
distro==1.7.0
dj-database-url==0.5.0
dj-pagination==2.5.0
django-allauth==0.47.0
django-appconf==1.0.3
django-autocomplete-light==3.5.1
django-bootstrap3-datetimepicker-2==2.8.2
django-celery-beat==2.2.1
django-celery-results==2.0.0
django-cors-headers==3.11.0
django-downloadview==2.3.0
django-filter==2.4.0
django-floppyforms==1.9.0
django-forms-bootstrap==3.1.0
django-geonode-mapstore-client==3.3.2
django-grappelli==2.15.1
django-guardian==2.4.0
django-haystack==3.1.1
django-imagekit==4.1.0
django-invitations==1.9.3
django-ipware==4.0.0
django-js-asset==1.2.2
django-jsonfield-backport==1.0.2
django-leaflet==0.27.1
django-markdownify==0.9.1
django-model-utils==0.0.0
django-modeltranslation==0.17.5
django-mptt==0.13.2
django-polymorphic==3.1.0
django-recaptcha==2.0.6
django-storages==1.12.3
django-taggit==2.1.0
django-tastypie==0.14.4
django-timezone-field==5.0
django-tinymce==3.3.0
django-treebeard==4.5.1
django-user-agents==0.4.0
django-uuid-upload-path==1.0.0
django-widget-tweaks==1.4.8
djangorestframework-gis==0.18
djangorestframework-guardian==0.3.0
djangorestframework==3.12.4
dnspython==2.1.0
docker==5.0.3
drf-extensions==0.7.1
drf-spectacular==0.21.1
drf-writable-nested==0.6.3
dynamic-rest==2.0.0
elasticsearch==7.16.2
entrypoints==0.4
ephem==4.1.3
factory-boy==2.11.1
flake8==4.0.1
flaky==3.7.0
folium==0.0.0
fonttools==4.29.1
frozenlist==1.2.0
fs==2.4.12
fsspec==2022.1.0
future==0.18.2
galternatives==1.0.8
geographiclib==1.52
geoip2==2.9.0
geojson==2.5.0
geolinks==0.2.0
geonode-announcements==2.0.2
geonode-avatar==5.0.7
geonode-dialogos==3.0.1
geonode-django-activity-stream==0.10.0
geonode-oauth-toolkit==2.2.2
geonode-user-messages==2.0.2
geopandas==0.10.2
geoserver-restconfig==2.0.7
gisdata==0.5.4
glob2==0.5
gn-arcrest==10.5.4
gn-gsimporter==2.0.1
gps==3.22
greenlet==1.1.2
gunicorn==20.1.0
hiredis==1.0.1
html5lib==1.1
httplib2==0.20.2
humanize==0.0.0
hyperlink==21.0.0
idna==3.3
importlib-metadata==4.6.4
incremental==21.3.0
inflection==0.3.1
iniconfig==1.1.1
invoke==1.4.1
iotop==0.6
ipykernel==6.7.0
ipyleaflet==0.17.0
ipython==7.31.1
ipython_genutils==0.2.0
ipywidgets==6.0.0
iso8601==1.0.2
isodate==0.6.1
itsdangerous==2.1.0
jdcal==1.0
jedi==0.18.0
jeepney==0.7.1
jmespath==0.10.0
joblib==0.17.0
jsonfield==3.1.0
jsonref==0.1
jsonschema==3.2.0
jupyter-client==7.1.2
jupyter-core==4.9.1
jupyter-packaging==0.11.1
jupyterlab-pygments==0.1.2
jwcrypto==1.0
keyring==23.5.0
kiwisolver==1.3.2
kombu==5.2.3
language-selector==0.1
lark-parser==0.12.0
launchpadlib==1.10.16
lazr.restfulclient==0.14.4
lazr.uri==1.0.6
locket==0.2.1
lubuntu-update-notifier==0.4.dev0
lxml==4.8.0
lz4==3.1.3+dfsg
mapnik==3.0.23
mappyfile==0.9.7
mapscript==7.6.4
matplotlib-inline==0.1.3
matplotlib==3.5.1
maxminddb==2.0.3
mccabe==0.6.1
mercantile==1.1.5
mock==4.0.3
more-itertools==8.10.0
mpmath==0.0.0
msal-extensions==1.0.0
msal==1.17.0
msgpack==1.0.3
msrest==0.6.21
msrestazure==0.6.4
multidict==5.1.0
munch==2.5.0
nbclient==0.5.6
nbconvert==6.4.0
nbformat==5.1.3
nest-asyncio==1.5.4
netCDF4==1.5.8
netifaces==0.11.0
networkx==2.4
notebook==6.4.8
numpy==1.21.5
oauthlib==3.2.0
osmapi==3.0.0
osmium==3.2.0
packaging==21.3
palettable==3.3.0
pandas==1.3.5
pandocfilters==1.5.0
paramiko==2.9.3
parse-type==0.5.6
parse==1.19.0
parso==0.8.1
partd==1.2.0
pbr==5.8.0
pexpect==4.8.0
pickleshare==0.7.5
pilkit==2.0
pinax-notifications==6.0.0
pinax-ratings==4.0.0
plotly==5.4.0
pluggy==0.13.0
ply==3.11
portalocker==2.2.1
prometheus-client==0.9.0
prompt-toolkit==3.0.28
psutil==5.9.0
psycopg2==2.8.6
ptyprocess==0.7.0
py==1.10.0
pyOpenSSL==21.0.0
pyasn1-modules==0.2.1
pyasn1==0.4.8
pycairo==1.20.1
pycodestyle==2.8.0
pycountry==20.7.3
pycql==0.0.8
pycsw==2.6.1
pycups==2.0.1
pydantic==1.8.2
pyflakes==2.4.0
pygeoapi==0.12.0
pylibmc==1.5.2
pyows==0.2.6
pyparsing==2.4.7
pyproj==3.3.0
pyrsistent==0.18.1
pysal==2.1.0
pyserial==3.5
pyshp==2.2.0
pytest-bdd==3.2.1
pytest-django==3.5.1
pytest==6.2.5
python-apt==2.3.0+ubuntu2
python-crontab==2.6.0
python-dateutil==2.8.1
python-debian===0.1.43ubuntu1
python-etcd==0.4.5
python-memcached==1.58
python-mimeparse==1.6.0
python-slugify==4.0.0
python3-openid==3.2.0
pytz==2022.1
pywps==4.5.1
pyxdg==0.27
pyzmq==22.3.0
rasterio==1.3.0
redis==3.5.3
requests-oauthlib==1.3.0
requests-toolbelt==0.9.1
requests==2.25.1
s3transfer==0.5.0
schema==0.6.7
scikit-learn==0.23.2
scipy==1.8.0
seaborn==0.11.2
selenium==4.0.0a1
service-identity==18.1.0
sherlock==0.3.2
six==1.16.0
smart-open==5.2.1
snuggs==1.4.7
sortedcontainers==2.1.0
soupsieve==2.3.1
sqlparse==0.4.2
sympy==1.9
systemd-python==234
tblib==1.7.0
tenacity==6.3.1
terminado==0.13.1
testpath==0.5.0
threadpoolctl==3.1.0
timeout-decorator==0.5.0
tinydb==3.15.2
toml==0.10.2
tomlkit==0.9.2
toolz==0.11.2
tornado==6.1
tqdm==4.57.0
traitlets==5.1.1
traittypes==0.2.1
typing-extensions==3.10.0.2
tzlocal==2.1
ua-parser==0.10.0
uamqp==1.5.1
ubuntu-advantage-tools==27.7
ubuntu-drivers-common==0.0.0
ufoLib2==0.13.1
ufw==0.36.1
unattended-upgrades==0.1
unicodecsv==0.14.1
unicodedata2==14.0.0
uritemplate==3.0.1
urllib3==1.26.5
usb-creator==0.3.7
user-agents==2.2.0
vine==5.0.0
wadllib==1.3.6
wcwidth==0.2.5
webencodings==0.5.1
websocket-client==1.2.3
websockets==9.1
widgetsnbextension==2.0.0
wrapt==1.13.3
wxPython==4.0.7
xarray==0.16.1
xdg==5
xkit==0.0.0
xmljson==0.1.9
xmltodict==0.12.0
xyzservices==2022.6.0
yarl==1.7.2
zict==2.0.0
zipp==1.0.0
zipstream==1.1.4
zope.interface==5.4.0


Focal Snapshot local

                Sat 26 Feb 2022 07:10:31 PM PST

$ ls .local/lib/python3.8/site-packages/ | grep info | sed -e 's/\.dist-info//'
aenum-3.1.8
affine-2.3.0
appdirs-1.4.4
bolt_python-0.7.1
boto-2.49.0
boto3-1.21.2
botocore-1.24.2
branca-0.4.2
charset_normalizer-2.0.12
dataclasses_json-0.5.6
Deprecated-1.2.13
eo_learn-1.0.0
eo_learn_core-1.0.0
eo_learn_coregistration-1.0.0
eo_learn_features-1.0.0
eo_learn_geometry-1.0.0
eo_learn_io-1.0.0
eo_learn_mask-1.0.0
eo_learn_ml_tools-1.0.0
eo_learn_visualization-1.0.0
folium-0.12.1.post1
fs-2.4.15
fs_s3fs-1.1.1
graphviz-0.19.1
imageio-2.16.0
ipyleaflet-0.15.0
ipywidgets-7.6.5
jmespath-0.10.0
joblib-1.1.0
jupyterlab_widgets-1.0.2
lightgbm-3.3.2
llvmlite-0.38.0
marshmallow-3.14.1
marshmallow_enum-1.5.1
mypy_extensions-0.4.3
networkx-2.6.3
numba-0.55.1
numpy-1.21.5
opencv_contrib_python_headless-4.5.5.62
openeo-0.9.2
pandas-1.4.1
Pillow-8.4.0
python_dateutil-2.8.2
pytz-2021.3
PyWavelets-1.2.0
rasterio-1.2.10
requests-2.27.1
requests_oauthlib-1.3.1
s3transfer-0.5.1
scikit_image-0.19.2
scikit_learn-1.0.2
scipy-1.8.0
sentinelhub-3.4.4
snuggs-1.4.7
threadpoolctl-3.1.0
thunder_python-1.4.2
thunder_registration-1.0.1
tifffile-2022.2.9
tqdm-4.62.3
traittypes-0.2.1
typing_extensions-4.1.1
typing_inspect-0.7.1
utm-0.7.0
widgetsnbextension-3.5.2
wrapt-1.13.3
xarray-0.21.1
xyzservices-2022.2.0
Retrieved from "http://wiki.osgeo.org/w/index.php?title=Osgeolive-jupyter&oldid=129924"