Installation

Create a virtual environment

It is recommended to install HELM into a virtual environment with Python version 3.8 to avoid dependency conflicts. HELM requires Python version 3.8. To create, a Python virtual environment with Python version >= 3.8 and activate it, follow the instructions below.

Using Virtualenv:

# Create a virtual environment.
# Only run this the first time.
python3 -m pip install virtualenv
python3 -m virtualenv -p python3.8 helm-venv

# Activate the virtual environment.
source helm-venv/bin/activate

Using Anaconda:

# Create a virtual environment.
# Only run this the first time.
conda create -n crfm-helm python=3.8 pip

# Activate the virtual environment.
conda activate crfm-helm

Install HELM

Within this virtual environment, run:

pip install crfm-helm

For HEIM (text-to-image evaluation)

To install the additional dependencies to run HEIM, run:

pip install "crfm-helm[heim]"

Some models (e.g., DALLE-mini/mega) and metrics (DetectionMetric) require extra dependencies that are not available on PyPI. To install these dependencies, download and run the extra install script:

bash install-heim-extras.sh