Osgeolive-jupyter

From OSGeo
Revision as of 16:06, 24 February 2022 by Darkblueb (talk | contribs) (→‎R setup)
Jump to navigation Jump to search

Focal Notes

Use PPA ubuntugis-unstable and default Ubuntu Universe dot-deb packages, the installation results look roughly like this:

dbb@focalk:/etc$ cat lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04.3 LTS"
#--
 PGSQL="120" GEOS="3.10.1-CAPI-1.16.0" PROJ="8.2.0" LIBXML="2.9.10" LIBJSON="0.13.1"
#--

                                    Sat 19 Feb 2022 07:18:48 PM PST
$ jupyter --version
jupyter core     : 4.6.3
jupyter-notebook : 6.0.3
qtconsole        : not installed
ipython          : 7.13.0
ipykernel        : 5.2.0
jupyter client   : 6.1.2
jupyter lab      : not installed
nbconvert        : 5.6.1
ipywidgets       : 6.0.0
nbformat         : 5.0.4
traitlets        : 4.3.3

##-------------------------------
Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]
pandas version: 0.25.3
matplotlib version: 3.1.2
NumPy version: 1.17.4
SciPy version: 1.3.3
IPython version: 7.13.0
scikit-learn version: 1.0.2
##-----------------------------------------
dbb@focalk:~$ ipython3
Python 3.8.10 (default, Nov 26 2021, 20:14:08) 
Type 'copyright', 'credits' or 'license' for more information
IPython 7.13.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: import sklearn                                                                                               
In [2]: from sklearn import show_versions                                                                            
In [3]: show_versions()                                                                                              

System:
    python: 3.8.10 (default, Nov 26 2021, 20:14:08)  [GCC 9.3.0]
executable: /usr/bin/python3
   machine: Linux-5.13.0-30-generic-x86_64-with-glibc2.29

Python dependencies:
          pip: 20.0.2
   setuptools: 45.2.0
      sklearn: 1.0.2
        numpy: 1.17.4
        scipy: 1.3.3
       Cython: 0.29.14
       pandas: 0.25.3      <== a weak spot in this setup
   matplotlib: 3.1.2
       joblib: 1.1.0
threadpoolctl: 3.1.0

Built with OpenMP: True
--
dbb@focalk:~$ jupyter --paths
config:
    /home/dbb/.jupyter
    /usr/local/etc/jupyter
    /etc/jupyter
data:
    /home/dbb/.local/share/jupyter
    /usr/local/share/jupyter
    /usr/share/jupyter
runtime:
    /home/dbb/.local/share/jupyter/runtime
--

Focal Jupyter Overview

Ipsum Lorem etc...

Install Strategy

APT install into the system OS python, major, stable parts first.. as a set .. ; next

pip install --user  --upgrade-strategy=only-if-needed  py3-pkg

installs to the current user's local area `~/.local/lib/python3.8/` ; next define directory `~/.jupyter/notebooks/` ; add Jupyter notebooks & assets there; will change. This opinionated install will be updated/amended/evolving as installer code and more dot-debs are crafted later this year.

Some priority python for the Jupyter Notebooks of 2022

  • GeoPandas (and pySAL) : direct link to GEOS 3.10, popular extension to Pandas data science; please recall that pySAL is merging into the GeoPandas project overall; GeoPandas+pySAL use matplotlib graphics base, prefers Folium for web maps (?)
  • Numpy & Pandas : fundamental base packaging, versions matter
  • PostGIS writing done with pyscopg2 (low-level) or geoalchemy2 (ORM based on sqlalchemy)
  • Scikit-Learn : original machine-learning library by international research team led by French math; stands the tests of time; integrated with SciPy for Fortran, BLAS, LAPACK performant libs; standard numpy for data allocation; important primitives like PCA, SVM, and KNN are optimized in practical ways, and then benchmarked against pure Fortran or C++ for comparison
  • boto for AWS S3, other data store transfer; can use AWS credentials (or Hetzner, or..) modern version using pip installed locally
  • rasterio : modern geoTIFF handling; sophisticated library design; not easy to build from source
  • Shapely & Fiona : direct link to libGEOS/ogr from python; example Notebooks in place
  • mapclassify : part of pySAL, versions must match with geopandas setup
  • lightgbm : MSFT python, tricky to install, might have to wait until other versions are sorted here
  • libosmium2-dev, libosmium2-doc, python3-pyosmium, osmium-tool : a C++ framework for working with OSM data files; osmium library bindings for Python 3
  • libmapnik3.1, python3-mapnik : mapnik rendering engine for OSM maps and others
  • LiDAR "no" : this is slightly painful since modern commercial GIS is flooded with this tech, delivered by advanced Western companies seeking something novel to sell. Little to no "libre" support and that includes here.
  • DeepLearning "no" : deep learning has taken over in the modern AI/ML research, for funding and hype, but not on this disk. Please note that the ML libraries on osgeolive are completely useful, accurate and performant today.. no need for a constant and fast Internet connection, nor a compute infrastructure the size of a large naval ship, or special impenetrable models that can only be built by those machines. -LINK- Serious and in-demand applications like "facial recognition" are not included here. Many real applications to remote sensing are built using DeepLearning, but those are not available on this disk. Some benefits of not including DeepLearning : no dependencies on tricky GPU libs; not suffering from fast-updates of important ML core libraries; avoid black-box training models, and their disk space; avoid software patent problems in Western Nations.

"I feel that a common and well-known distro like this is an opportunity to build engineering cooperation, between teams that may not interact otherwise.. like the physical code sprints" -darkblueb in California

Example Notebooks in previous OSGeoLive -LINK-

Focal Setup Steps

  ## note gdal & postgis are installed at an earlier step via package dot-deb
  591  sudo apt install python3-dev python3-pip
  405  sudo apt install ipython3 python3-scipy
            apt install python3-pandas  ## <== stale version currently
  408  sudo apt install r-cran-sf       ## match libgeos w/R language bindings
  429  sudo apt install python3-shapely python3-fiona
  614  sudo apt install --yes python3-geoalchemy2
  ##-- show the parts explicitly for Jupyter on Ubuntu via dot-deb ---------------
  670  sudo apt install --yes jupyter-core jupyter-client jupyter-console jupyter-nbformat jupyter-nbconvert  \
                              jupyter-notebook python3-ipykernel
  673  sudo apt install python-notebook-doc jupyter
  ##--
       sudo apt install python3-sklearn-lib  python3-pyosmium

# use this form  $ pip install --user --upgrade-strategy=only-if-needed  expy3pkg

  605  pip install --user wheel
  606  pip install --user lightgbm
  630  pip install --user folium
  835  pip install --user --upgrade  --upgrade-strategy=only-if-needed pandas 
  836  pip install --user --upgrade  --upgrade-strategy=only-if-needed scikit-image
  837  pip install --user --upgrade  --upgrade-strategy=only-if-needed imageio
  838  pip install --user --upgrade  --upgrade-strategy=only-if-needed numba
  839  pip install --user --upgrade  --upgrade-strategy=only-if-needed sentinelhub
  841  pip install --user --upgrade  --upgrade-strategy=only-if-needed eo-learn
  853  pip install --user --upgrade  --upgrade-strategy=only-if-needed      ipyleaflet
  867  pip install --user --upgrade  --upgrade-strategy=only-if-needed    openeo

all [ rasterio, networkx, sklearn, boto, more...  ] into   ~/.local/lib/python3.8/  (below)
$ ./do_jupyter.sh
#---------------------------
#!/bin/bash

jupyter notebook --no-browser --ip=192.168.1.131 --notebook-dir=.jupyter/notebooks

~/.bashrc

##-
PATH=$PATH:~/.local/bin

Focal APT Repos

dbb@focalk:/etc/apt$ cat sources.list
# Automatically generated by Calamares on 2022-01-02.
# Lines starting with "deb" are mandatory, while lines starting with "deb-src"
# are for more detailed package information.

## See http://help.ubuntu.com/community/UpgradeNotes for how to upgrade to
## newer versions of Lubuntu.
deb http://ubuntu.osuosl.org/ubuntu/ focal main restricted
deb-src http://ubuntu.osuosl.org/ubuntu/ focal main restricted

## Major bug fix updates produced after the final release of Lubuntu.
## Have you noticed a regression? Please report it!
## https://wiki.ubuntu.com/StableReleaseUpdates#Regressions

## Software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu team.
## Also, please note that software in Universe WILL NOT receive any review or
## updates from the Ubuntu security team directly. Updates in this repository
## are provided by volunteers, but most come from Debian.
deb http://ubuntu.osuosl.org/ubuntu/ focal universe
deb-src http://ubuntu.osuosl.org/ubuntu/ focal universe

## Software from this repository is ENTIRELY UNSUPPORTED by the Ubuntu team,
## and may not be under a free licence. Please satisfy yourself as your rights
## to use the software. Also, please note that software in Multiverse WILL NOT
## receive any review or updates from the Ubuntu security team directly.
# deb http://us.archive.ubuntu.com/ubuntu/ focal multiverse
# deb-src http://us.archive.ubuntu.com/ubuntu/ focal multiverse
# deb http://us.archive.ubuntu.com/ubuntu/ focal-updates multiverse
# deb-src http://us.archive.ubuntu.com/ubuntu/ focal-updates multiverse

## Software from this repository contains tested security updates from the
## Ubuntu security team.
deb http://security.ubuntu.com/ubuntu focal-security main restricted
deb-src http://security.ubuntu.com/ubuntu focal-security main restricted
deb http://security.ubuntu.com/ubuntu focal-security universe
deb-src http://security.ubuntu.com/ubuntu focal-security universe
deb http://security.ubuntu.com/ubuntu focal-security multiverse
# deb-src http://security.ubuntu.com/ubuntu focal-security multiverse

## Software from this repository may not have been tested as extensively as
## software contained in the main release, although it includes newer versions
## of some applications which may provide useful features. Also, please note
## that software in Backports WILL NOT receive any review or updates from the
## Ubuntu security team.
deb http://ubuntu.osuosl.org/ubuntu/ focal-backports main restricted universe multiverse
deb-src http://ubuntu.osuosl.org/ubuntu/ focal-backports main restricted universe multiverse

## Uncomment the following two lines to add software from Canonical's
## "partner" repository.
## This software is not part of Ubuntu, but is offered by Canonical and the
## respective vendors as a service to Ubuntu users.
# deb http://archive.canonical.com/ubuntu focal partner
# deb-src http://archive.canonical.com/ubuntu focal partner

-------------------------

dbb@focalk:/etc/apt$ cat sources.list.d/ubuntugis-ubuntu-ubuntugis-unstable-focal.list

deb http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main
deb-src http://ppa.launchpad.net/ubuntugis/ubuntugis-unstable/ubuntu focal main


pip Fix-It Session eo-learn


$ pip install --user --upgrade  --upgrade-strategy=only-if-needed    pandas
Collecting pandas
  Downloading pandas-1.4.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (11.7 MB)
     |                                | 11.7 MB 79 kB/s 
Collecting pytz>=2020.1
  Downloading pytz-2021.3-py2.py3-none-any.whl (503 kB)
     |                                | 503 kB 260 kB/s 
Collecting python-dateutil>=2.8.1
  Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
     |                                | 247 kB 210 kB/s 
Collecting numpy>=1.18.5; platform_machine != "aarch64" and platform_machine != "arm64" and python_version < "3.10"
  Downloading numpy-1.22.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (16.8 MB)
     |                                | 16.8 MB 11 kB/s 
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil>=2.8.1->pandas) (1.14.0)
ERROR: sentinelhub 3.4.4 has requirement pillow<=8.4.0,>=8.3.2, but you'll have pillow 7.0.0 which is incompatible.
ERROR: scikit-image 0.19.2 has requirement scipy>=1.4.1, but you'll have scipy 1.3.3 which is incompatible.
ERROR: numba 0.55.1 has requirement numpy<1.22,>=1.18, but you'll have numpy 1.22.2 which is incompatible.
ERROR: imageio 2.16.0 has requirement pillow>=8.3.2, but you'll have pillow 7.0.0 which is incompatible.
Installing collected packages: pytz, python-dateutil, numpy, pandas
Successfully installed numpy-1.22.2 pandas-1.4.1 python-dateutil-2.8.2 pytz-2021.3

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed     scikit-image
Requirement already up-to-date: scikit-image in ./.local/lib/python3.8/site-packages (0.19.2)
Requirement already satisfied, skipping upgrade: imageio>=2.4.1 in ./.local/lib/python3.8/site-packages (from scikit-image) (2.16.0)
Requirement already satisfied, skipping upgrade: packaging>=20.0 in /usr/lib/python3/dist-packages (from scikit-image) (20.3)
Collecting scipy>=1.4.1
  Downloading scipy-1.8.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (41.6 MB)
     |                                | 41.6 MB 20 kB/s 
Requirement already satisfied, skipping upgrade: pillow!=7.1.0,!=7.1.1,!=8.3.0,>=6.1.0 in /usr/lib/python3/dist-packages (from scikit-image) (7.0.0)
Requirement already satisfied, skipping upgrade: numpy>=1.17.0 in ./.local/lib/python3.8/site-packages (from scikit-image) (1.22.2)
Requirement already satisfied, skipping upgrade: PyWavelets>=1.1.1 in ./.local/lib/python3.8/site-packages (from scikit-image) (1.2.0)
Requirement already satisfied, skipping upgrade: networkx>=2.2 in ./.local/lib/python3.8/site-packages (from scikit-image) (2.6.3)
Requirement already satisfied, skipping upgrade: tifffile>=2019.7.26 in ./.local/lib/python3.8/site-packages (from scikit-image) (2022.2.9)
Installing collected packages: scipy
Successfully installed scipy-1.8.0

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      imageio
Requirement already up-to-date: imageio in ./.local/lib/python3.8/site-packages (2.16.0)
Requirement already satisfied, skipping upgrade: numpy>=1.20.0 in ./.local/lib/python3.8/site-packages (from imageio) (1.22.2)
Collecting pillow>=8.3.2
  Downloading Pillow-9.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (4.3 MB)
     |                                | 4.3 MB 99 kB/s 
ERROR: sentinelhub 3.4.4 has requirement pillow<=8.4.0,>=8.3.2, but you'll have pillow 9.0.1 which is incompatible.
Installing collected packages: pillow
Successfully installed pillow-9.0.1

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      numba
Requirement already up-to-date: numba in ./.local/lib/python3.8/site-packages (0.55.1)
Collecting numpy<1.22,>=1.18
  Downloading numpy-1.21.5-cp38-cp38-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (15.7 MB)
     |                                | 15.7 MB 361 kB/s 
Requirement already satisfied, skipping upgrade: setuptools in /usr/lib/python3/dist-packages (from numba) (45.2.0)
Requirement already satisfied, skipping upgrade: llvmlite<0.39,>=0.38.0rc1 in ./.local/lib/python3.8/site-packages (from numba) (0.38.0)
ERROR: sentinelhub 3.4.4 has requirement pillow<=8.4.0,>=8.3.2, but you'll have pillow 9.0.1 which is incompatible.
Installing collected packages: numpy
  Attempting uninstall: numpy
    Found existing installation: numpy 1.22.2
    Uninstalling numpy-1.22.2:
      Successfully uninstalled numpy-1.22.2
Successfully installed numpy-1.21.5

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed       sentinelhub
Requirement already up-to-date: sentinelhub in ./.local/lib/python3.8/site-packages (3.4.4)
Requirement already satisfied, skipping upgrade: oauthlib in /usr/lib/python3/dist-packages (from sentinelhub) (3.1.0)
Requirement already satisfied, skipping upgrade: requests-oauthlib in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.3.1)
Requirement already satisfied, skipping upgrade: dataclasses-json in ./.local/lib/python3.8/site-packages (from sentinelhub) (0.5.6)
Requirement already satisfied, skipping upgrade: numpy in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.21.5)
Requirement already satisfied, skipping upgrade: pyproj>=2.2.0 in /usr/lib/python3/dist-packages (from sentinelhub) (3.2.1)
Requirement already satisfied, skipping upgrade: python-dateutil in ./.local/lib/python3.8/site-packages (from sentinelhub) (2.8.2)
Requirement already satisfied, skipping upgrade: requests>=2.5.0 in /usr/lib/python3/dist-packages (from sentinelhub) (2.22.0)
Requirement already satisfied, skipping upgrade: tifffile in ./.local/lib/python3.8/site-packages (from sentinelhub) (2022.2.9)
Requirement already satisfied, skipping upgrade: utm in ./.local/lib/python3.8/site-packages (from sentinelhub) (0.7.0)
Collecting pillow<=8.4.0,>=8.3.2
  Downloading Pillow-8.4.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (3.1 MB)
     |                                | 3.1 MB 43 kB/s 
Requirement already satisfied, skipping upgrade: click in /usr/lib/python3/dist-packages (from sentinelhub) (7.0)
Requirement already satisfied, skipping upgrade: aenum>=2.1.4 in ./.local/lib/python3.8/site-packages (from sentinelhub) (3.1.8)
Requirement already satisfied, skipping upgrade: boto3 in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.21.2)
Requirement already satisfied, skipping upgrade: shapely in /usr/lib/python3/dist-packages (from sentinelhub) (1.8.0)
Requirement already satisfied, skipping upgrade: botocore in ./.local/lib/python3.8/site-packages (from sentinelhub) (1.24.2)
Requirement already satisfied, skipping upgrade: tqdm in ./.local/lib/python3.8/site-packages (from sentinelhub) (4.62.3)
Requirement already satisfied, skipping upgrade: typing-inspect>=0.4.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub) (0.7.1)
Requirement already satisfied, skipping upgrade: marshmallow-enum<2.0.0,>=1.5.1 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub) (1.5.1)
Requirement already satisfied, skipping upgrade: marshmallow<4.0.0,>=3.3.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub) (3.14.1)
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil->sentinelhub) (1.14.0)
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in ./.local/lib/python3.8/site-packages (from boto3->sentinelhub) (0.10.0)
Requirement already satisfied, skipping upgrade: s3transfer<0.6.0,>=0.5.0 in ./.local/lib/python3.8/site-packages (from boto3->sentinelhub) (0.5.1)
Requirement already satisfied, skipping upgrade: urllib3<1.27,>=1.25.4 in /usr/lib/python3/dist-packages (from botocore->sentinelhub) (1.25.8)
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub) (4.1.1)
Requirement already satisfied, skipping upgrade: mypy-extensions>=0.3.0 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub) (0.4.3)
Installing collected packages: pillow
  Attempting uninstall: pillow
    Found existing installation: Pillow 9.0.1
    Uninstalling Pillow-9.0.1:
      Successfully uninstalled Pillow-9.0.1
Successfully installed pillow-8.4.0

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      eo-learn
Requirement already up-to-date: eo-learn in ./.local/lib/python3.8/site-packages (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-io>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-coregistration>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-features>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-mask>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-ml-tools>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-core>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-visualization>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: eo-learn-geometry>=1.0.0 in ./.local/lib/python3.8/site-packages (from eo-learn) (1.0.0)
Requirement already satisfied, skipping upgrade: rasterio>=1.2.7 in ./.local/lib/python3.8/site-packages (from eo-learn-io>=1.0.0->eo-learn) (1.2.10)
Requirement already satisfied, skipping upgrade: sentinelhub>=3.4.0 in ./.local/lib/python3.8/site-packages (from eo-learn-io>=1.0.0->eo-learn) (3.4.4)
Requirement already satisfied, skipping upgrade: fiona>=1.8.18 in /usr/lib/python3/dist-packages (from eo-learn-io>=1.0.0->eo-learn) (1.8.20)
Requirement already satisfied, skipping upgrade: rtree in /usr/lib/python3/dist-packages (from eo-learn-io>=1.0.0->eo-learn) (0.9.4)
Requirement already satisfied, skipping upgrade: geopandas>=0.8.1 in /usr/lib/python3/dist-packages (from eo-learn-io>=1.0.0->eo-learn) (0.10.2)
Requirement already satisfied, skipping upgrade: boto3 in ./.local/lib/python3.8/site-packages (from eo-learn-io>=1.0.0->eo-learn) (1.21.2)
Requirement already satisfied, skipping upgrade: opencv-contrib-python-headless>=4.1 in ./.local/lib/python3.8/site-packages (from eo-learn-coregistration>=1.0.0->eo-learn) (4.5.5.62)
Requirement already satisfied, skipping upgrade: thunder-registration in ./.local/lib/python3.8/site-packages (from eo-learn-coregistration>=1.0.0->eo-learn) (1.0.1)
Requirement already satisfied, skipping upgrade: python-dateutil in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (2.8.2)
Requirement already satisfied, skipping upgrade: scipy in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (1.8.0)
Requirement already satisfied, skipping upgrade: numba>=0.53.0 in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (0.55.1)
Requirement already satisfied, skipping upgrade: numpy in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (1.21.5)
Requirement already satisfied, skipping upgrade: scikit-learn in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (1.0.2)
Requirement already satisfied, skipping upgrade: scikit-image>=0.19.0 in ./.local/lib/python3.8/site-packages (from eo-learn-features>=1.0.0->eo-learn) (0.19.2)
Requirement already satisfied, skipping upgrade: lightgbm>=2.0.11 in ./.local/lib/python3.8/site-packages (from eo-learn-mask>=1.0.0->eo-learn) (3.3.2)
Requirement already satisfied, skipping upgrade: shapely in /usr/lib/python3/dist-packages (from eo-learn-ml-tools>=1.0.0->eo-learn) (1.8.0)
Requirement already satisfied, skipping upgrade: attrs>=19.2.0 in /usr/lib/python3/dist-packages (from eo-learn-core>=1.0.0->eo-learn) (19.3.0)
Requirement already satisfied, skipping upgrade: tqdm>=4.27 in ./.local/lib/python3.8/site-packages (from eo-learn-core>=1.0.0->eo-learn) (4.62.3)
Requirement already satisfied, skipping upgrade: fs in ./.local/lib/python3.8/site-packages (from eo-learn-core>=1.0.0->eo-learn) (2.4.15)
Requirement already satisfied, skipping upgrade: fs-s3fs in ./.local/lib/python3.8/site-packages (from eo-learn-core>=1.0.0->eo-learn) (1.1.1)
Requirement already satisfied, skipping upgrade: pydot in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (1.4.1)
Requirement already satisfied, skipping upgrade: jinja2 in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (2.10.1)
Requirement already satisfied, skipping upgrade: matplotlib in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (3.1.2)
Requirement already satisfied, skipping upgrade: graphviz>=0.10.1 in ./.local/lib/python3.8/site-packages (from eo-learn-visualization>=1.0.0->eo-learn) (0.19.1)
Requirement already satisfied, skipping upgrade: pygments in /usr/lib/python3/dist-packages (from eo-learn-visualization>=1.0.0->eo-learn) (2.3.1)
Requirement already satisfied, skipping upgrade: descartes in /usr/lib/python3/dist-packages (from eo-learn-geometry>=1.0.0->eo-learn) (1.1.0)
Requirement already satisfied, skipping upgrade: certifi in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (2019.11.28)
Requirement already satisfied, skipping upgrade: setuptools in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (45.2.0)
Requirement already satisfied, skipping upgrade: click-plugins in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (1.1.1)
Requirement already satisfied, skipping upgrade: cligj>=0.5 in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (0.5.0)
Requirement already satisfied, skipping upgrade: affine in ./.local/lib/python3.8/site-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (2.3.0)
Requirement already satisfied, skipping upgrade: snuggs>=1.4.1 in ./.local/lib/python3.8/site-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (1.4.7)
Requirement already satisfied, skipping upgrade: click>=4.0 in /usr/lib/python3/dist-packages (from rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (7.0)
Requirement already satisfied, skipping upgrade: pyproj>=2.2.0 in /usr/lib/python3/dist-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.2.1)
Requirement already satisfied, skipping upgrade: oauthlib in /usr/lib/python3/dist-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.1.0)
Requirement already satisfied, skipping upgrade: aenum>=2.1.4 in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.1.8)
Requirement already satisfied, skipping upgrade: dataclasses-json in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.5.6)
Requirement already satisfied, skipping upgrade: requests>=2.5.0 in /usr/lib/python3/dist-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (2.22.0)
Requirement already satisfied, skipping upgrade: tifffile in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (2022.2.9)
Requirement already satisfied, skipping upgrade: utm in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.7.0)
Requirement already satisfied, skipping upgrade: requests-oauthlib in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.3.1)
Requirement already satisfied, skipping upgrade: pillow<=8.4.0,>=8.3.2 in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (8.4.0)
Requirement already satisfied, skipping upgrade: botocore in ./.local/lib/python3.8/site-packages (from sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.24.2)
Requirement already satisfied, skipping upgrade: jmespath<1.0.0,>=0.7.1 in ./.local/lib/python3.8/site-packages (from boto3->eo-learn-io>=1.0.0->eo-learn) (0.10.0)
Requirement already satisfied, skipping upgrade: s3transfer<0.6.0,>=0.5.0 in ./.local/lib/python3.8/site-packages (from boto3->eo-learn-io>=1.0.0->eo-learn) (0.5.1)
Requirement already satisfied, skipping upgrade: thunder-python>=1.2.0 in ./.local/lib/python3.8/site-packages (from thunder-registration->eo-learn-coregistration>=1.0.0->eo-learn) (1.4.2)
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/lib/python3/dist-packages (from python-dateutil->eo-learn-features>=1.0.0->eo-learn) (1.14.0)
Requirement already satisfied, skipping upgrade: llvmlite<0.39,>=0.38.0rc1 in ./.local/lib/python3.8/site-packages (from numba>=0.53.0->eo-learn-features>=1.0.0->eo-learn) (0.38.0)
Requirement already satisfied, skipping upgrade: threadpoolctl>=2.0.0 in ./.local/lib/python3.8/site-packages (from scikit-learn->eo-learn-features>=1.0.0->eo-learn) (3.1.0)
Requirement already satisfied, skipping upgrade: joblib>=0.11 in ./.local/lib/python3.8/site-packages (from scikit-learn->eo-learn-features>=1.0.0->eo-learn) (1.1.0)
Requirement already satisfied, skipping upgrade: PyWavelets>=1.1.1 in ./.local/lib/python3.8/site-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (1.2.0)
Requirement already satisfied, skipping upgrade: imageio>=2.4.1 in ./.local/lib/python3.8/site-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (2.16.0)
Requirement already satisfied, skipping upgrade: networkx>=2.2 in ./.local/lib/python3.8/site-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (2.6.3)
Requirement already satisfied, skipping upgrade: packaging>=20.0 in /usr/lib/python3/dist-packages (from scikit-image>=0.19.0->eo-learn-features>=1.0.0->eo-learn) (20.3)
Requirement already satisfied, skipping upgrade: wheel in /usr/lib/python3/dist-packages (from lightgbm>=2.0.11->eo-learn-mask>=1.0.0->eo-learn) (0.34.2)
Requirement already satisfied, skipping upgrade: pytz in ./.local/lib/python3.8/site-packages (from fs->eo-learn-core>=1.0.0->eo-learn) (2021.3)
Requirement already satisfied, skipping upgrade: appdirs~=1.4.3 in ./.local/lib/python3.8/site-packages (from fs->eo-learn-core>=1.0.0->eo-learn) (1.4.4)
Requirement already satisfied, skipping upgrade: pyparsing>=2.1.6 in /usr/lib/python3/dist-packages (from snuggs>=1.4.1->rasterio>=1.2.7->eo-learn-io>=1.0.0->eo-learn) (2.4.6)
Requirement already satisfied, skipping upgrade: typing-inspect>=0.4.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.7.1)
Requirement already satisfied, skipping upgrade: marshmallow<4.0.0,>=3.3.0 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (3.14.1)
Requirement already satisfied, skipping upgrade: marshmallow-enum<2.0.0,>=1.5.1 in ./.local/lib/python3.8/site-packages (from dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.5.1)
Requirement already satisfied, skipping upgrade: urllib3<1.27,>=1.25.4 in /usr/lib/python3/dist-packages (from botocore->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (1.25.8)
Requirement already satisfied, skipping upgrade: boto>=2.36.0 in ./.local/lib/python3.8/site-packages (from thunder-python>=1.2.0->thunder-registration->eo-learn-coregistration>=1.0.0->eo-learn) (2.49.0)
Requirement already satisfied, skipping upgrade: bolt-python>=0.7.0 in ./.local/lib/python3.8/site-packages (from thunder-python>=1.2.0->thunder-registration->eo-learn-coregistration>=1.0.0->eo-learn) (0.7.1)
Requirement already satisfied, skipping upgrade: mypy-extensions>=0.3.0 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (0.4.3)
Requirement already satisfied, skipping upgrade: typing-extensions>=3.7.4 in ./.local/lib/python3.8/site-packages (from typing-inspect>=0.4.0->dataclasses-json->sentinelhub>=3.4.0->eo-learn-io>=1.0.0->eo-learn) (4.1.1)

$ pip install --user --upgrade  --upgrade-strategy=only-if-needed      ipyleaflet
Collecting ipyleaflet
  Downloading ipyleaflet-0.15.0-py2.py3-none-any.whl (3.3 MB)
     |                                | 3.3 MB 364 kB/s 
Collecting traittypes<3,>=0.2.1
  Downloading traittypes-0.2.1-py2.py3-none-any.whl (8.6 kB)
Collecting xyzservices>=2021.8.1
  Downloading xyzservices-2022.2.0-py3-none-any.whl (35 kB)
Collecting ipywidgets<8,>=7.6.0
  Downloading ipywidgets-7.6.5-py2.py3-none-any.whl (121 kB)
     |                                | 121 kB 338 kB/s 
Requirement already satisfied, skipping upgrade: traitlets>=4.2.2 in /usr/lib/python3/dist-packages (from traittypes<3,>=0.2.1->ipyleaflet) (4.3.3)
Requirement already satisfied, skipping upgrade: ipykernel>=4.5.1 in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (5.2.0)
Requirement already satisfied, skipping upgrade: ipython>=4.0.0; python_version >= "3.3" in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (7.13.0)
Requirement already satisfied, skipping upgrade: ipython-genutils~=0.2.0 in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (0.2.0)
Collecting jupyterlab-widgets>=1.0.0; python_version >= "3.6"
  Downloading jupyterlab_widgets-1.0.2-py3-none-any.whl (243 kB)
     |                                | 243 kB 376 kB/s 
Collecting widgetsnbextension~=3.5.0
  Downloading widgetsnbextension-3.5.2-py2.py3-none-any.whl (1.6 MB)
     |                                | 1.6 MB 359 kB/s 
Requirement already satisfied, skipping upgrade: nbformat>=4.2.0 in /usr/lib/python3/dist-packages (from ipywidgets<8,>=7.6.0->ipyleaflet) (5.0.4)
Requirement already satisfied, skipping upgrade: pexpect in /usr/lib/python3/dist-packages (from ipython>=4.0.0; python_version >= "3.3"->ipywidgets<8,>=7.6.0->ipyleaflet) (4.6.0)
Requirement already satisfied, skipping upgrade: notebook>=4.4.1 in /usr/lib/python3/dist-packages (from widgetsnbextension~=3.5.0->ipywidgets<8,>=7.6.0->ipyleaflet) (6.0.3)
Installing collected packages: traittypes, xyzservices, jupyterlab-widgets, widgetsnbextension, ipywidgets, ipyleaflet
Successfully installed ipyleaflet-0.15.0 ipywidgets-7.6.5 jupyterlab-widgets-1.0.2 traittypes-0.2.1 widgetsnbextension-3.5.2 xyzservices-2022.2.0

##jupyter nbextension enable --py --sys-prefix{?} ipyleaflet  # can be skipped for notebook 5.3 and above
 eolearn - first aprox. of all depends python
import 
:  boto3
:  cv2
:  fiona
:  graphviz
:  geopandas as gpd
:  pandas as pd
:  rasterio
:  rasterio.features
:  rasterio.transform
:  rasterio.warp
:  shapely.geometry
:  shapely.ops
:  shapely.wkt
:  skimage.feature
:  skimage.filters.rank
:  skimage.morphology
:  skimage.segmentation
: ------------------------------------------
:  abc
:  attr
:  base64
:  collections
:  concurrent.futures
:  dataclasses
:  datetime as dt
:  dateutil
:  dateutil.parser
:  fs
:  functools
:  gzip
:  importlib
:  inspect
:  itertools as it
:  json
:  logging
:  multiprocessing
:  os
:  pickle
:  ray
:  re
:  registration
:  tempfile
:  threading
:  time
:  traceback
:  unittest.mock as mock
:  uuid
:  warnings
: --------------------------


$ ls .local/lib/python3.8/site-packages/ | grep info | sed -e 's/\.dist-info//'
aenum-3.1.8
affine-2.3.0
appdirs-1.4.4
bolt_python-0.7.1
boto-2.49.0
boto3-1.21.2
botocore-1.24.2
branca-0.4.2
charset_normalizer-2.0.12
dataclasses_json-0.5.6
Deprecated-1.2.13
eo_learn-1.0.0
eo_learn_core-1.0.0
eo_learn_coregistration-1.0.0
eo_learn_features-1.0.0
eo_learn_geometry-1.0.0
eo_learn_io-1.0.0
eo_learn_mask-1.0.0
eo_learn_ml_tools-1.0.0
eo_learn_visualization-1.0.0
folium-0.12.1.post1
fs-2.4.15
fs_s3fs-1.1.1
graphviz-0.19.1
imageio-2.16.0
ipyleaflet-0.15.0
ipywidgets-7.6.5
jmespath-0.10.0
joblib-1.1.0
jupyterlab_widgets-1.0.2
lightgbm-3.3.2
llvmlite-0.38.0
marshmallow-3.14.1
marshmallow_enum-1.5.1
mypy_extensions-0.4.3
networkx-2.6.3
numba-0.55.1
numpy-1.21.5
opencv_contrib_python_headless-4.5.5.62
openeo-0.9.2
pandas-1.4.1
Pillow-8.4.0
python_dateutil-2.8.2
pytz-2021.3
PyWavelets-1.2.0
rasterio-1.2.10
requests-2.27.1
requests_oauthlib-1.3.1
s3transfer-0.5.1
scikit_image-0.19.2
scikit_learn-1.0.2
scipy-1.8.0
sentinelhub-3.4.4
snuggs-1.4.7
threadpoolctl-3.1.0
thunder_python-1.4.2
thunder_registration-1.0.1
tifffile-2022.2.9
tqdm-4.62.3
traittypes-0.2.1
typing_extensions-4.1.1
typing_inspect-0.7.1
utm-0.7.0
widgetsnbextension-3.5.2
wrapt-1.13.3
xarray-0.21.1
xyzservices-2022.2.0


R setup

Focal default R

  $ R version 3.6.3 (2020-02-29) -- "Holding the Windsock"

A better future awaits: https://cran.r-project.org/bin/linux/ubuntu/

-- R language example from aigeolabs.com Explainable Machine Learning for Land Cover Classification

# Load the libraries
rm(list=ls())
library(sf)    # represents simple features as native R objects
library(rgdal) # interface to the “Geospatial Data Abstraction Library (GDAL)
library(raster) # provides important functions for importing and handling raster data
library(rasterVis) # provides functions to display raster objects
library(dplyr) # provides basic data transformation functions
library(ggplot2) # provides extension for visualizations
library(mapview) # provides functions to visualize geospatial data
library(randomForest) # implements Breiman’s random forest (RF) algorithm
library(kernlab) # implements support vector machine (SVM) algorithm
library(corrplot) # provides graphical display of a correlation matrix
library(iml)   # Keras-base tools for analyzing machine learning models
library(caret) #  machine learning helper

##--------------
$ apt install r-base-core --yes
  ...
The following NEW packages will be installed:
  gfortran gfortran-9 libgfortran-9-dev libpcre2-32-0 libpcre2-dev libpcre2-posix2 r-base-core r-base-dev
  r-cran-boot r-cran-class r-cran-cluster r-cran-codetools r-cran-foreign r-cran-kernsmooth r-cran-lattice
  r-cran-mass r-cran-matrix r-cran-mgcv r-cran-nlme r-cran-nnet r-cran-rpart r-cran-spatial r-cran-survival
  r-doc-html r-recommended

$ apt install r-cran-sf --yes
  ...
The following NEW packages will be installed:
  fonts-glewlwyd libjs-d3 libjs-es5-shim libjs-highlight.js libjs-jquery-datatables
  libjs-jquery-datatables-extensions libjs-jquery-selectize.js libjs-json libjs-microplugin.js libjs-prettify
  libjs-sifter.js libjs-twitter-bootstrap-datepicker libnode64 libudunits2-0 libudunits2-data littler
  node-highlight.js node-html5shiv node-normalize.css nodejs nodejs-doc r-cran-abind r-cran-askpass
  r-cran-assertthat r-cran-backports r-cran-base64enc r-cran-bh r-cran-bit r-cran-bit64 r-cran-bitops r-cran-blob
  r-cran-brew r-cran-callr r-cran-classint r-cran-cli r-cran-cliapp r-cran-clipr r-cran-clisymbols
  r-cran-colorspace r-cran-commonmark r-cran-covr r-cran-crayon r-cran-crosstalk r-cran-curl r-cran-data.table
  r-cran-dbi r-cran-dbitest r-cran-deldir r-cran-desc r-cran-devtools r-cran-diffobj r-cran-digest r-cran-domc
  r-cran-dplyr r-cran-dt r-cran-e1071 r-cran-ellipsis r-cran-evaluate r-cran-fansi r-cran-farver r-cran-fastmap
  r-cran-filehash r-cran-fontbitstreamvera r-cran-fontliberation r-cran-fontquiver r-cran-foreach
  r-cran-freetypeharfbuzz r-cran-fs r-cran-future r-cran-gdtools r-cran-generics r-cran-ggplot2 r-cran-gh
  r-cran-git2r r-cran-globals r-cran-glue r-cran-goftest r-cran-gsl r-cran-gtable r-cran-hexbin r-cran-highr
  r-cran-hms r-cran-htmltools r-cran-htmlwidgets r-cran-httpuv r-cran-httr r-cran-ini r-cran-iterators
  r-cran-jsonlite r-cran-knitr r-cran-labeling r-cran-later r-cran-lazyeval r-cran-lifecycle r-cran-listenv
  r-cran-littler r-cran-lubridate r-cran-magrittr r-cran-mapproj r-cran-maps r-cran-maptools r-cran-markdown
  r-cran-memoise r-cran-mime r-cran-mockery r-cran-mockr r-cran-multicore r-cran-munsell r-cran-openssl
  r-cran-pillar r-cran-pkgbuild r-cran-pkgconfig r-cran-pkgkitten r-cran-pkgload r-cran-plogr r-cran-plyr
  r-cran-png r-cran-polyclip r-cran-praise r-cran-prettycode r-cran-prettyunits r-cran-processx r-cran-progress
  r-cran-promises r-cran-ps r-cran-purrr r-cran-r.methodss3 r-cran-r.oo r-cran-r6 r-cran-randomfields
  r-cran-randomfieldsutils r-cran-raster r-cran-rcmdcheck r-cran-rcolorbrewer r-cran-rcpp r-cran-remotes
  r-cran-reshape2 r-cran-rex r-cran-rgdal r-cran-rlang r-cran-rmarkdown r-cran-roxygen2 r-cran-rpostgresql
  r-cran-rprojroot r-cran-rsqlite r-cran-rstudioapi r-cran-runit r-cran-rversions r-cran-scales r-cran-selectr
  r-cran-sessioninfo r-cran-sf r-cran-shiny r-cran-sodium r-cran-sourcetools r-cran-sp r-cran-spatstat
  r-cran-spatstat.data r-cran-spatstat.utils r-cran-stringi r-cran-stringr r-cran-sys r-cran-systemfonts
  r-cran-tensor r-cran-testit r-cran-testthat r-cran-tibble r-cran-tidyr r-cran-tidyselect r-cran-tikzdevice
  r-cran-tinytex r-cran-units r-cran-usethis r-cran-utf8 r-cran-vctrs r-cran-vdiffr r-cran-viridislite
  r-cran-webutils r-cran-whisker r-cran-withr r-cran-xfun r-cran-xml r-cran-xml2 r-cran-xopen r-cran-xtable
  r-cran-yaml
0 upgraded, 186 
Warning from rgdal
  ...
> library(rgdal)
Loading required package: sp
rgdal: version: 1.5-18, (SVN revision 1082)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 3.4.0, released 2021/11/04
Path to GDAL shared files: /usr/share/gdal
GDAL binary built with GEOS: TRUE 
Loaded PROJ runtime: Rel. 8.2.0, November 1st, 2021, [PJ_VERSION: 820]
Path to PROJ shared files: /home/dbb/.local/share/proj:/usr/share/proj
PROJ CDN enabled: FALSE
Linking to sp version:1.4-0
To mute warnings of possible GDAL/OSR exportToProj4() degradation,
use options("rgdal_show_exportToProj4_warnings"="none") before loading rgdal.

OSGeoLive install_R.sh ver14

apt-get --assume-yes install r-cran-sp r-cran-rpostgresql r-cran-udunits2 \
  r-cran-ggplot2 r-cran-sf r-cran-lwgeom r-cran-spdata r-cran-rgdal r-cran-maptools