For the Installation of Software in Offline Environments
(and those with restrictive outbound )
Please don't expect too much from the site or us "we are just doing what we can".
We genuinely would like to hear from people with new techniques and software not covered. Please email hello.world@offlinesetup.com
Bioconductor
As suggested in their documentation.
rsync -zrtlv --exclude 'bioc/bin/macosx' --exclude 'bioc/bin/windows/contrib/3*' --delete master.bioconductor.org::release/ /repo1/bioconductor/release/
Conda (free/pro/main/cond-forge/r(python))
The coda repository is several TB in size. We decided to only mirror python version 3.9..
We quickly found out that only mirroring packages with py39 resulted in missing dependency's. We had to adopt the method of excluding packages with other version in their name,
noarch is needed.
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://repo.anaconda.com/pkgs/pro/linux-64/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://repo.anaconda.com/pkgs/pro/noarch/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)" https://repo.anaconda.com/pkgs/pro/win-64/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://repo.anaconda.com/pkgs/free/linux-64/wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*
)" https://repo.anaconda.com/pkgs/free/noarch/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)"https://repo.anaconda.com/pkgs/free/win-64/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*
)" https://repo.anaconda.com/pkgs/main/linux-64/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*
)" https://repo.anaconda.com/pkgs/main/noarch/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)" https://repo.anaconda.com/pkgs/main/win-64/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://conda.anaconda.org/conda-forge/linux-64/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://conda.anaconda.org/conda-forge/noarch/
wget -m -np -nH --cut-dirs=1 -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)" https://conda.anaconda.org/conda-forge/win-64/
wget -m -np -nH -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://conda.anaconda.org/pytorch/linux-64/
wget -m -np -nH -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://conda.anaconda.org/pytorch/win-64/
wget -m -np -nH -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://conda.anaconda.org/pytorch/noarch/
wget -m -np -nH -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://repo.anaconda.com/pkgs/r/linux-64/
wget -m -np -nH -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://repo.anaconda.com/pkgs/r/win-64/
wget -m -np -nH -R "index.html*" -e robots=off --reject-regex "(.*py2[123456789].*)|(.*py3[12345678].*)|(.*-arm64.*)|(.*ppc64.*)|(.*aarch64.*)" https://repo.anaconda.com/pkgs/r/noarch/
INLA
There probably is a better way of creating a mirror than this.
wget -m -np -nH -e robots=off -R --regex-type pcre https://inla.r-inla-download.org/R/stable/src/contrib/
wget -m -np -nH -e robots=off -R --regex-type pcre https://inla.r-inla-download.org/R/stable/bin/windows/contrib/
Linux YUM Repositories.
RedHat, CentOS(RIP). Oracle Linux, Rocky Linux ,etc.
You can easily create your own mirror of publicly available repositories. For RedHat OS repositories you need a licenced (paid for) account to connect to each version of RedHat repo.
With "enterprise 8" the process is much easier than previous versions.
With enterprise 8 you can mirror repositories for versions 6 and 7.
Check the repositories you have installed using the command
yum repolist
Add the additional repositories to the Linux OS
For RedHat repositories you will need to use subscription-manager such as ..
subscription-manager repos --enable=codeready-builder-for-rhel-8-x86_64-rpms
For non-RedHat repositories (even on RedHat OS) just add a conf file to /etc/yum.repod.d/
As an example add the following to /etc/yum.repos.d/OL8_CR_64.conf
[ol8_codeready_builder]
name=Oracle Linux 8 CodeReady Builder ($basearch) - Unsupported
baseurl=https://yum.oracle.com/repo/OracleLinux/OL8/codeready/builder/x86_64/
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-oracle
gpgcheck=1
enabled=1
Another Example (as requested) will be /etc/yum.repos.d/defender.conf
[packages-microsoft-com-prod]
name=packages-microsoft-com-prod
baseurl=https://packages.microsoft.com/rhel/8.2/prod/
enabled=1
gpgcheck=1
gpgkey=https://packages.microsoft.com/keys/microsoft.asc
One significant gotcha is when it comes to updating the OS of the mirror itself . It will attempt to update installed packages with the the versions in repositories you have installed. Obviously this installing packages from Oracle Linux repositories on a RedHat Linux will result in a poorly OS. You need something similar to
yum --disablerepo="*" --enablerepo="rhel-8-for-x86_64-appstream-rpms" --enablerepo="rhel-8-for-x86_64-baseos-rpms" --enablerepo="packages-microsoft-com-prod" update
After all that, let's remind ourselves that all we have done is make the OS aware of the repos we wish to mirror. To create the mirrors use
dnf reposync --download-metadata -p /repo/yum_repos/ --remote-time
R (CRAN) Repo
As suggested in t he R documentation
rsync -rtlzv --delete --exclude 'bin/linux/ubuntu' --exclude 'bin/macosx' --exclude 'bin/macos' --exclude 'bin/linux/debian' cran.r-project.org::CRAN /repo/R/
You may wish to exclude more directories. The R repository keeps a lot of old versions.
The repository includes the R installation itself and rtools
Visual Studio Code Extensions
There is no automated method we have found.
Just go the marketplace and select "Download Extension".
We have found that with a small number of packages that we have "compatibility issues", and sometimes we have to download previous extension versions (yeah, it does not make sense)
When the extensions don't work........
We have found that those previous versions are not advertised and there is often no indication which version you have to download.
Oh enough of this, we are talking "Jupyter",
The method we use is to have a machine on the internet with the same VSC. Install the extension from the market place (not a vsix file) and make a note of the version.
Also make a note of any dependencies that may have been installed! You will need them,
Now go back to the market place for the extension you need and a click on "Version History"
Right click on any of the downloads and copy the URL.
Edit the URL replacing the version number with the one you made a note of..
You should be able to download the VSIX file for a working extensions.
If this works (which it should) let us know by going to offlinesetup.com - Contact