Hi,
First of all, apologies if I’m asking any stupid question. I’m pretty new to conda.
I’ve been trying for a couple of days to successfully install pillow-simd
and libjpeg-turbo
in a conda environment to reduce the time to load images and doing some data augmentations. At times I’ve been successful but I’m not able to make it reproducible.
I generally create the conda environment using the following command:
conda create --name my_env python=3.7 pandas numpy scipy matplotlib jupyterlab xgboost lightgbm catboost pytorch torchvision torchaudio cudatoolkit=11.0 rapids-blazing=0.17 pytorch-lightning dvc kaggle black mypy dask albumentations -c pytorch -c rapidsai -c nvidia -c conda-forge -c defaults
and then run the following script, as suggested in the fastai documentation.
conda uninstall -y --force pillow pil pillow-simd jpeg libtiff libjpeg-turbo
pip uninstall -y pillow pil pillow-simd jpeg libtiff libjpeg-turbo
conda install -yc conda-forge libjpeg-turbo
CFLAGS="${CFLAGS} -mavx2" pip install --upgrade --no-cache-dir --force-reinstall --no-binary :all: --compile pillow-simd==7.0.0.post3
conda install -y jpeg libtiff
I then check if pillow-simd
is using libjpeg-turbo
by using the following script:
import PIL
from PIL import features
print(f"PIL version: {PIL.__version__}")
if features.check_feature('libjpeg_turbo'):
print("libjpeg-turbo is on")
else:
print("libjpeg-turbo is NOT on")
Unfortunately, at times I end up having a system with:
PIL version: 8.1.0
libjpeg-turbo is NOT on
Other times I have pillow=7.0.0
installed, but it doesn’t use libjpeg-turbo
. Other times it works.
The problem seems to be that libjpeg-turbo
requires pillow=8.1.0
. To fix that, I’ve changed the command to install libjpeg-turbo
to:
conda install -yc conda-forge libjpeg-turbo=2.0.3 pillow=7.0.0
… but this makes the script to verify if PIL uses libjpeg-turbo fails with this error message:
ImportError: The _imaging extension was built for another version of Pillow or PIL:
Core version: 7.0.0.post3
Pillow version: 7.0.0
Has anyone found a robust way to install pillow-simd
and libjpeg-turbo
alongside other data science libraries?