Your cart is empty.
Keep shopping
Teach on Udemy
Turn what you know into an opportunity and reach millions around the world.
Learn MoreSoftware package management with conda
Rating: 3.8 out of 53.8(7 ratings)
47 students
Software package management with conda
Use conda to install software, create separate software environment, delete software
Created byCristian Riccio
Last updated 10/2022
English
What you'll learn
- conda software package management
- installing Anaconda
- install software with conda
- create separate conda environments
- integrate conda with snakemake
- control software versions with conda
- learn about conda channels
Explore related topics
Course content
13 sections • 39 lectures • 1h 46m total length
- Getting started with conda0:03
You will find everything you need to know about conda at the following link:
https://conda.io/projects/conda/en/latest/user-guide/getting-started.html#
- Data Camp resources for learning conda0:02
- Install Anaconda1:21
Find more detailed information here: https://docs.continuum.io/anaconda/install/mac-os/
Go to the Home directory:
cd
Download the command-line installer for your operating system and architecture (32-bit vs 64-bit):
Example on Mac OS: wget https://repo.anaconda.com/archive/Anaconda3-2020.11-MacOSX-x86_64.sh
Example on GNU/Linux: wget https://repo.anaconda.com/archive/Anaconda3-2020.11-Linux-x86_64.sh
Install for Python 3.7:
Example on Mac OS: bash Anaconda3-2020.11-MacOSX-x86_64.sh
Example on GNU/Linux: bash Anaconda3-2020.11-Linux-x86_64.sh
The installer prompts “In order to continue the installation process, please review the license agreement.” Click Enter to view the license terms.
Scroll to the bottom of the license terms (for example by pressing the space bar) and enter yes to agree to them.
The installer prompts you to Press Enter to confirm the location, Press CTRL-C to cancel the installation or specify an alternate installation directory. If you confirm the default location, it will display PREFIX=/home/<user>/anaconda<2 or 3> and continue the installation.
At the final step, it will give you the option to append anaconda’s installation location to your system’s PATH variable. Choose 'yes' so that you will not need to specify the complete path to the conda commands (id est you will just need to type 'conda install package_name' instead of typing '/path/to/the/conda/executable/conda install package_name'.
Close and open your terminal window for the Anaconda installation to take effect.
To run conda from anywhere without having the base environment activated by default, use:
conda config --set auto_activate_base False
This only works if you have run conda init first.
Verify your installation: https://docs.continuum.io/anaconda/install/verify-install/
After opening the terminal, choose any of the following methods to verify:
Enter conda list. If Anaconda is installed and working, this will display a list of installed packages and their versions.
conda activate base. Enter the command python. This command runs the Python shell. If Anaconda is installed and working, the version information it displays when it starts up will include “Anaconda”. To exit the Python shell, enter the command quit().
conda activate base. Open Anaconda Navigator with the command anaconda-navigator. If Anaconda is installed properly, Anaconda Navigator will open.
After the installation has successfully finished, you can delete the installer:
Example on Mac OS: rm Anaconda3-2020.11-MacOSX-x86_64.sh
Example on GNU/Linux: rm Anaconda3-2020.11-Linux-x86_64.sh
Here I show you how to install Anaconda. Note: Anaconda can take a while to install. For example, it can remain for a longer time at "Executing transaction...". So leave it overnight before becoming impatient and thinking something is wrong.
- Install Anaconda using the Graphical User Interface1:09
Install Anaconda using a graphical user interface and learn more about Anaconda and the conda package manager.
- Uninstall Anaconda0:17
- Install Miniconda0:41
In this article, I explain why you might need to install a smaller version of Anaconda called Miniconda. I also point you to resources where to find the Miniconda installers and instructions on how to install Miniconda.
- What is conda?
- conda --version or conda -V: find the version of your conda installation0:04
To find the version of your conda installation, in your terminal, type:
conda --version
or
conda -V
but NOT:
conda -v
- Find the version of your conda installation
- conda info: get information on your conda installation1:39
Get information on your conda installation.
In your terminal, type:
conda info
- conda: package dependencies, why are they important and how to find them?12:48
- Where are conda packages installed?0:05
Conda packages are installed by default in:
~/anaconda3/envs/
For example, sed version 4.8 is installed at
~/anaconda3/envs/sed=4.8/bin/sed
You can test it by typing:
~/anaconda3/envs/sed=4.8/bin/sed -h
- Search for a conda package1:43
Search for a conda package in a conda channel
conda search --channel CHANNELNAME samtools
e.g.
conda search --channel bioconda samtools
- Search for a conda package on a specific operating system (platform)12:00
Search for a conda package in a conda channel
conda search --channel CHANNELNAME samtools
e.g.
conda search --channel bioconda samtools
With the above commands, only packages available on your computer platform will be shown.
Search for a conda package in a conda channel on an operating system different from your own:
conda search --channel CHANNELNAME --platform PLATFORM PACKAGE_NAME
conda search --channel bioconda --platform linux-64 cactus
The "platform" option can take different values. The main ones are linux-64, osx-64, win-64, linux-32 and win-32. Other platforms include:
linux-aarch64
linux-armv6l
linux-armv7l
linux-ppc64
linux-ppc64le
linux-s390x
osx-arm64
- Search for a conda package on ALL channels at the same time5:56
Search for a package in all channels and all platforms simultaneously
Install anaconda-client
Type:
anaconda search PACKAGE_NAME
Example:
anaconda search survivor
- conda install: install a conda package4:03
- conda install: install several conda packages simultaneously2:34
- conda create: create a conda environment3:10
conda create --name ENV_NAME --channel CHANNEL_NAME --yes PACKAGE_NAME=PACKAGE_VERSION
Example:
conda create --name samtools=1.11 --channel bioconda --yes samtools=1.11
- Create a conda environment from a YAML file0:01
conda env create -f <path_to_yaml_file>
- conda activate and conda deactivate: activate and deactivate conda environments3:02
- conda info --envs: list conda environments1:50
- How do I find out which environment is currently active?4:39
- Which conda environment is currently active?
- Install a conda package in the active environment3:27
- Export a conda environment to a YAML file0:05
While the environment is active:
conda env export > environment.yml
Now the environment.yaml file contains the specifications (packages + versions) of the conda environment.
- conda env remove: remove a conda environment1:09
- conda channels: where conda packages live8:02
- bioconda: the conda channel for bioinformatics software0:14
Bioconda is the conda channel for bioinformatics software. A lot of the tools I use in bioinformatics live in the bioconda channel: samtools, pilon, vcftools, etc. Find out more at:
https://www.nature.com/articles/s41592-018-0046-7
https://rdcu.be/chkFw
Grüning, B., Dale, R., Sjödin, A. et al. Bioconda: sustainable and comprehensive software distribution for the life sciences. Nat Methods 15, 475–476 (2018). https://doi.org/10.1038/s41592-018-0046-7
Published02 July 2018
Issue DateJuly 2018
DOIhttps://doi.org/10.1038/s41592-018-0046-7
- List conda channels and their priorities0:41
When my ~/.condarc looks like this:
channels:
- defaults
- conda-forge
- bioconda
ssl_verify: true
The following command:
conda config --get channels
returns:
--add channels 'bioconda' # lowest priority
--add channels 'conda-forge'
--add channels 'defaults' # highest priority
When my ~/.condarc looks like this:
channels:
- bioconda
- conda-forge
- defaults
ssl_verify: true
The following command:
conda config --get channels
returns:
--add channels 'defaults' # lowest priority
--add channels 'conda-forge'
--add channels 'bioconda' # highest priority
This is not a coincidence. The packages listed at the top in the ~/.condarc configuration file have the highest priority. This means that if conda is looking to install package "X", it will first look into the channel with the highest priority. If it does not find it there, it will look in the channel with the second-highest priority, and so on. Of course, as we have seen previously, it is best practice to not rely on the default priorities. Instead, the channels to look into should be specified with each relevant command.
For example, when installing samtools, run:
conda install --channel bioconda samtools
This way, conda will first look into the bioconda channel for a package called "samtools".
- condarc: configure conda to add channels to be searched by default4:54
- Find the version of a conda package in your active environment0:07
conda list PACKAGE_NAME
Example:
conda list r-base
Output:
# packages in environment at /Users/user/anaconda3/envs/main:
#
# Name Version Build Channel
r-base 4.0.3 h280a0cc_2 conda-forge
Therefore, the version of the package "r-base" in the currently active environment is 4.0.3.
- conda list REGEX: find the version of conda packages using regular expressions6:23
conda list 'REGULAR_EXPRESSION'
Examples:
conda list 'r-dplyr|r-ggplot2'
lists the versions of the packages 'r-dplyr' and 'r-ggplot2'
conda list 'r-'
lists the versions of all the packages whose names contain the string 'r-'
conda lists '^r-'
lists the versions of all the packages whose names start (^) with the string 'r-'.
conda list 'a$'
lists the versions of all the packages whose names end ($) with the letter 'a'.
- conda list: find the version of conda packages in a non-active environment4:25
- conda update: update a conda package to its latest version3:08
conda update PACKAGE_NAME
Example:
conda update r-dplyr
It is possible to update more than one package at a time:
conda update PACKAGE_NAME1 PACKAGE_NAME2
Example:
conda update r-dplyr r-ggplot2 r-tidyr
- To update conda to the newest version, type the following in your terminal: con3:16
To update conda to the newest version, type the following in your terminal:
conda update conda
If your conda version needs updating, you will be prompted to accept the update. In your terminal, type:
y
for "yes".
It is recommended to regularly update conda to the latest version.
conda install --channel conda-forge --yes r-covr=3.5.1
Collecting package metadata (current_repodata.json): done
Solving environment: done
==> WARNING: A newer version of conda exists. <==
current version: 4.10.1
latest version: 4.10.2
Please update conda by running
$ conda update -n base conda
# All requested packages already installed.
(r-all) ➜ Documents conda update -n base conda
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /Users/cristianriccio/anaconda3
added / updated specs:
- conda
The following packages will be downloaded:
package | build
---------------------------|-----------------
alabaster-0.7.12 | py_0 15 KB conda-forge
applaunchservices-0.2.1 | py_0 8 KB conda-forge
async_generator-1.10 | py_0 18 KB conda-forge
backports.tempfile-1.0 | py_0 10 KB conda-forge
backports.weakref-1.0.post1|py38h32f6830_1002 8 KB conda-forge
beautifulsoup4-4.9.3 | pyhb0f4dca_0 86 KB conda-forge
chardet-3.0.4 | py38_1003 170 KB conda-forge
cloudpickle-1.6.0 | py_0 22 KB conda-forge
conda-4.10.2 | py38h50d1736_0 3.1 MB conda-forge
conda-package-handling-1.7.3| py38h96a0964_0 1.5 MB conda-forge
dask-2.30.0 | py_0 4 KB conda-forge
decorator-4.4.2 | py_0 11 KB conda-forge
flake8-3.8.4 | py_0 89 KB conda-forge
freetype-2.10.4 | ha233b18_0 890 KB conda-forge
fsspec-0.8.3 | py_0 64 KB conda-forge
glob2-0.7 | py_0 11 KB conda-forge
heapdict-1.0.1 | py_0 7 KB conda-forge
imageio-2.9.0 | py_0 3.1 MB conda-forge
imagesize-1.2.0 | py_0 8 KB conda-forge
importlib_metadata-2.0.0 | 1 3 KB conda-forge
itsdangerous-1.1.0 | py_0 16 KB conda-forge
jsonschema-3.2.0 | py_2 45 KB conda-forge
olefile-0.46 | py_0 31 KB conda-forge
path.py-12.5.0 | 0 4 KB conda-forge
pathtools-0.1.2 | py_1 8 KB conda-forge
pluggy-0.13.1 | py38_0 29 KB conda-forge
python_abi-3.8 | 2_cp38 4 KB conda-forge
qtpy-1.9.0 | py_0 34 KB conda-forge
sphinxcontrib-applehelp-1.0.2| py_0 28 KB conda-forge
sphinxcontrib-devhelp-1.0.2| py_0 22 KB conda-forge
sphinxcontrib-jsmath-1.0.1 | py_0 7 KB conda-forge
sphinxcontrib-qthelp-1.0.3 | py_0 25 KB conda-forge
threadpoolctl-2.1.0 | pyh5ca1d4c_0 15 KB conda-forge
tk-8.6.10 | hb0a8c7a_0 3.3 MB conda-forge
toolz-0.11.1 | py_0 46 KB conda-forge
urllib3-1.25.11 | py_0 93 KB conda-forge
widgetsnbextension-3.5.1 | py38_0 1.8 MB conda-forge
xlrd-1.2.0 | py_0 108 KB conda-forge
xmltodict-0.12.0 | py_0 11 KB conda-forge
zict-2.0.0 | py_0 10 KB conda-forge
------------------------------------------------------------
Total: 14.7 MB
The following packages will be UPDATED:
backports.functoo~ pkgs/main::backports.functools_lru_ca~ --> conda-forge::backports.functools_lru_cache-1.6.4-pyhd8ed1ab_0
backports.weakref pkgs/main/noarch::backports.weakref-1~ --> conda-forge/osx-64::backports.weakref-1.0.post1-py38h32f6830_1002
conda 4.10.1-py38h50d1736_0 --> 4.10.2-py38h50d1736_0
conda-package-han~ pkgs/main::conda-package-handling-1.7~ --> conda-forge::conda-package-handling-1.7.3-py38h96a0964_0
python_abi 3.8-1_cp38 --> 3.8-2_cp38
The following packages will be SUPERSEDED by a higher-priority channel:
alabaster pkgs/main --> conda-forge
applaunchservices pkgs/main --> conda-forge
async_generator pkgs/main --> conda-forge
backports pkgs/main --> conda-forge
backports.tempfile pkgs/main::backports.tempfile-1.0-py_1 --> conda-forge::backports.tempfile-1.0-py_0
beautifulsoup4 pkgs/main --> conda-forge
chardet pkgs/main --> conda-forge
cloudpickle pkgs/main --> conda-forge
dask pkgs/main --> conda-forge
decorator pkgs/main --> conda-forge
et_xmlfile pkgs/main --> conda-forge
flake8 pkgs/main --> conda-forge
freetype pkgs/main --> conda-forge
fsspec pkgs/main --> conda-forge
glob2 pkgs/main --> conda-forge
heapdict pkgs/main --> conda-forge
imageio pkgs/main --> conda-forge
imagesize pkgs/main --> conda-forge
importlib_metadata pkgs/main --> conda-forge
itsdangerous pkgs/main --> conda-forge
jdcal pkgs/main --> conda-forge
jsonschema pkgs/main --> conda-forge
olefile pkgs/main --> conda-forge
path.py pkgs/main --> conda-forge
pathtools pkgs/main --> conda-forge
pluggy pkgs/main --> conda-forge
python-dateutil pkgs/main --> conda-forge
qtpy pkgs/main --> conda-forge
sphinxcontrib-app~ pkgs/main --> conda-forge
sphinxcontrib-dev~ pkgs/main --> conda-forge
sphinxcontrib-jsm~ pkgs/main --> conda-forge
sphinxcontrib-qth~ pkgs/main --> conda-forge
threadpoolctl pkgs/main --> conda-forge
tk pkgs/main --> conda-forge
toolz pkgs/main --> conda-forge
traitlets pkgs/main --> conda-forge
urllib3 pkgs/main --> conda-forge
widgetsnbextension pkgs/main --> conda-forge
xlrd pkgs/main --> conda-forge
xmltodict pkgs/main --> conda-forge
yaml pkgs/main --> conda-forge
zict pkgs/main --> conda-forge
Proceed ([y]/n)? y
Downloading and Extracting Packages
itsdangerous-1.1.0 | 16 KB | #################################################################################### | 100%
glob2-0.7 | 11 KB | #################################################################################### | 100%
async_generator-1.10 | 18 KB | #################################################################################### | 100%
chardet-3.0.4 | 170 KB | #################################################################################### | 100%
backports.tempfile-1 | 10 KB | #################################################################################### | 100%
olefile-0.46 | 31 KB | #################################################################################### | 100%
tk-8.6.10 | 3.3 MB | #################################################################################### | 100%
cloudpickle-1.6.0 | 22 KB | #################################################################################### | 100%
qtpy-1.9.0 | 34 KB | #################################################################################### | 100%
flake8-3.8.4 | 89 KB | #################################################################################### | 100%
importlib_metadata-2 | 3 KB | #################################################################################### | 100%
urllib3-1.25.11 | 93 KB | #################################################################################### | 100%
toolz-0.11.1 | 46 KB | #################################################################################### | 100%
heapdict-1.0.1 | 7 KB | #################################################################################### | 100%
pluggy-0.13.1 | 29 KB | #################################################################################### | 100%
jsonschema-3.2.0 | 45 KB | #################################################################################### | 100%
sphinxcontrib-jsmath | 7 KB | #################################################################################### | 100%
zict-2.0.0 | 10 KB | #################################################################################### | 100%
pathtools-0.1.2 | 8 KB | #################################################################################### | 100%
sphinxcontrib-qthelp | 25 KB | #################################################################################### | 100%
python_abi-3.8 | 4 KB | #################################################################################### | 100%
conda-package-handli | 1.5 MB | #################################################################################### | 100%
sphinxcontrib-appleh | 28 KB | #################################################################################### | 100%
beautifulsoup4-4.9.3 | 86 KB | #################################################################################### | 100%
backports.weakref-1. | 8 KB | #################################################################################### | 100%
imagesize-1.2.0 | 8 KB | #################################################################################### | 100%
path.py-12.5.0 | 4 KB | #################################################################################### | 100%
widgetsnbextension-3 | 1.8 MB | #################################################################################### | 100%
freetype-2.10.4 | 890 KB | #################################################################################### | 100%
xlrd-1.2.0 | 108 KB | #################################################################################### | 100%
xmltodict-0.12.0 | 11 KB | #################################################################################### | 100%
conda-4.10.2 | 3.1 MB | #################################################################################### | 100%
imageio-2.9.0 | 3.1 MB | #################################################################################### | 100%
dask-2.30.0 | 4 KB | #################################################################################### | 100%
decorator-4.4.2 | 11 KB | #################################################################################### | 100%
threadpoolctl-2.1.0 | 15 KB | #################################################################################### | 100%
alabaster-0.7.12 | 15 KB | #################################################################################### | 100%
sphinxcontrib-devhel | 22 KB | #################################################################################### | 100%
fsspec-0.8.3 | 64 KB | #################################################################################### | 100%
applaunchservices-0. | 8 KB | #################################################################################### | 100%
Preparing transaction: done
Verifying transaction: done
Executing transaction: / Enabling notebook extension jupyter-js-widgets/extension...
- Validating: OK
done
Requirements
- Basic Terminal/command line skills would be helpful
Description
conda is a package management system. It forms part of the basic toolkit of any programmer. It is especially useful in bioinformatics and for people who use a lot of different software and packages. It is also useful for R and Python programmers. I would recommend that you learn it at the beginning of your programmer's or bioinformatician's career. It has saved me many headaches and has vastly simplified software installation. It is also indispensable if you want to use software which have incompatible dependencies. This happens very often. For example, some software want Python 2 on your system, while others want Python 3. conda allows you to solve this kind of problems.
Use conda to:
- find software
- find software specific to your operating system
- install software
- install R packages
- install Python modules
- install bioinformatics software from the bioconda channel
- create separate software environment
- delete software
- remove conda environments
- change conda environment
- control software versions
- update software versions
- use incompatible software on the same computer
- use the .condarc file to configure conda
- use conda efficiently by adding channels to the .condarc file
- write reproducible and tidy code
- name your conda environments in a tidy, informative and reproducible way
For the time being, this course does not teach how to package your own software into conda packages. You do not need this course to learn conda. However, I this course teaches you conda step by step with ample narration and explanations. As more students join, I will add more videos and improve the quality but for now all the topics above are covered.
Who this course is for:
- programmers
- people new to programming
- data scientists
- biologists
- bioinformaticians
- people new to bioinformatics
- software pipeline managers
- science PhD students