Fastai on Apple M1

tcapelle · January 18, 2022, 1:55pm

Yeah, I am open to add more benchmarks if needed, and when Pytorch comes I will probably update the report.
My take is that the M1Pro (non Max) is a very descent GPU for light ML prototyping stuff, the max is just too expensive. Keep in mind that the M1 benchmarks are without any tune up like mixed precision, XLA, etc… just straight conda. Meanwhile, the Nvidia benchmarks are run in a highly optimised docker container from google.

Other take is that both apple cpus are way faster than colab and even Colab Pro.

Also, for 40 Watts you get 1/10th the performance of a 400 Watt RTX3090, so it’s pretty good.

mike.moloch · January 24, 2022, 8:46pm

Thanks for your reply, and thanks for doing that comparison!

I’m really amazed at this because I have scrounged together an old Xeon 8c/16t T3600 with 64G ram and a 1070ti and it is (barely) on par with Colab(with GPU enabled).

What this tells me is that I can replace that 2 ton machine with a mid tier M1-Pro or entry level M1Max MBP! … Except for any fastai related stuff, we probably need to wait for pytorch to catch up.

And re: the power to perf ratio, seems to me it’s giving decent performance when looked at from that perspective (perf/watt) … I’m very interested in knowing how Apple’s desktop/server level CPUs will fare.

dewanshu · May 19, 2022, 12:50am

Pytorch released support for Macbook m1 gpu on 18 may. You can try it out now.

Borz · May 19, 2022, 1:34am

Has anyone run fastai with the new pytorch nightly that supports M1 GPUs? I see this funny thing where if I pip-install it I can confirm it works (torch.backends.mps), but only if I uninstall fastai first. If I install fastai on top of it, maybe because of a torchvision requirement, pytorch loses the ability to check for M1 gpus.

gerardo · May 19, 2022, 1:49am

I just submitted an issue to the github repository.

dls = DataBlock(
blocks=(ImageBlock, CategoryBlock),
get_items=get_image_files,
splitter=RandomSplitter(valid_pct=0.2, seed=42),
get_y=parent_label,
item_tfms=[Resize(192, method=‘squish’)]
).dataloaders(path)

dls.show_batch(max_n=6)

github.com/fastai/fastai

AttributeError: '_FakeLoader' object has no attribute 'pin_memory_device'

opened 11:37PM - 18 May 22 UTC

geg00

Be sure you've searched [the forums](https://forums.fast.ai) for the error messa…ge you received. Also, unless you're an experienced fastai developer, first ask on the forums to see if someone else has seen a similar issue already and knows how to solve it. **Only file a bug report here when you're quite confident it's not an issue with your local setup.** **Please see [this model example](https://github.com/fastai/fastai2/issues/487) of how to fill out an issue correctly. Please try to emulate that example as appropriate when opening an issue.** Please confirm you have the latest versions of fastai, fastcore, and nbdev prior to reporting a bug (delete one): YES / NO **Describe the bug** AttributeError: '_FakeLoader' object has no attribute 'pin_memory_device' When I execute the dos.show_batch(max_n=6) **To Reproduce** Steps to reproduce the behavior: Successfully installed torch-1.12.0.dev20220511 Right after you execute the DataBlock section dls = DataBlock( blocks=(ImageBlock, CategoryBlock), get_items=get_image_files, splitter=RandomSplitter(valid_pct=0.2, seed=42), get_y=parent_label, item_tfms=[Resize(192, method='squish')] ).dataloaders(path) You execute the code dls.show_batch(max_n=6) ===> --------------------------------------------------------------------------- AttributeError Traceback (most recent call last) Untitled-1.ipynb Cell 9' in <cell line: 1>() ----> 1 dls.show_batch(max_n=6) File ~/opt/anaconda3/lib/python3.9/site-packages/fastai/data/core.py:102, in TfmdDL.show_batch(self, b, max_n, ctxs, show, unique, **kwargs) 100 old_get_idxs = self.get_idxs 101 self.get_idxs = lambda: Inf.zeros --> 102 if b is None: b = self.one_batch() 103 if not show: return self._pre_show_batch(b, max_n=max_n) 104 show_batch(*self._pre_show_batch(b, max_n=max_n), ctxs=ctxs, max_n=max_n, **kwargs) File ~/opt/anaconda3/lib/python3.9/site-packages/fastai/data/load.py:170, in DataLoader.one_batch(self) 168 def one_batch(self): 169 if self.n is not None and len(self)==0: raise ValueError(f'This DataLoader does not contain any batches') --> 170 with self.fake_l.no_multiproc(): res = first(self) 171 if hasattr(self, 'it'): delattr(self, 'it') 172 return res File ~/opt/anaconda3/lib/python3.9/site-packages/fastcore/basics.py:621, in first(x, f, negate, **kwargs) 619 x = iter(x) 620 if f: x = filter_ex(x, f=f, negate=negate, gen=True, **kwargs) --> 621 return next(x, None) File ~/opt/anaconda3/lib/python3.9/site-packages/fastai/data/load.py:125, in DataLoader.__iter__(self) 123 self.before_iter() 124 self.__idxs=self.get_idxs() # called in context of main process (not workers/subprocesses) --> 125 for b in _loaders[self.fake_l.num_workers==0](self.fake_l): 126 # pin_memory causes tuples to be converted to lists, so convert them back to tuples 127 if self.pin_memory and type(b) == list: b = tuple(b) 128 if self.device is not None: b = to_device(b, self.device) File ~/opt/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py:590, in _SingleProcessDataLoaderIter.__init__(self, loader) 589 def __init__(self, loader): --> 590 super(_SingleProcessDataLoaderIter, self).__init__(loader) 591 assert self._timeout == 0 592 assert self._num_workers == 0 File ~/opt/anaconda3/lib/python3.9/site-packages/torch/utils/data/dataloader.py:521, in _BaseDataLoaderIter.__init__(self, loader) 517 self._prefetch_factor = loader.prefetch_factor 518 # for other backends, pin_memory_device need to set. if not set 519 # default behaviour is CUDA device. if pin_memory_device is selected 520 # and pin_memory is not set, the default behaviour false. --> 521 if (len(loader.pin_memory_device) == 0): 522 self._pin_memory = loader.pin_memory and torch.cuda.is_available() 523 self._pin_memory_device = None AttributeError: '_FakeLoader' object has no attribute 'pin_memory_device' **Expected behavior** A clear and concise description of what you expected to happen. to show the images selected **Error with full stack trace** Place between these lines with triple backticks: ``` ``` **Additional context** I'm using the new version of torch-1.12.0.dev20220511 to take advantage of the M1 Metal option. Add any other context about the problem here.

mike.moloch · May 19, 2022, 2:21am

You can check some preliminary results @tcapelle has pubished today.

I ran that code on a Xeon+1070ti and have published my comparison in that thread as well.

radikubwa · May 19, 2022, 8:19am

PyTorch posted this. Maybe it would be useful here Introducing Accelerated PyTorch Training on Mac | PyTorch

pellet · May 23, 2022, 5:15am

I wanted to do some quick patching for proof of concept and seems that m1 is working with performance being around 2x what it was. The code should probably be rejigged to allow people to select MPS(m1) rather than CUDA if they want, but by default it can choose CUDA before MPS since generally speaking you’d get better performance on a CUDA card. Here is my quick and dirty patch to see the performance increase: Comparing fastai:master...pellet:master · fastai/fastai · GitHub

mike.moloch · May 23, 2022, 3:55pm

Some more numbers by reddit user u/seraschka (is it the Sebastian Raschka?)

rawmean · May 24, 2022, 10:36pm

Thank you for this patch. Can you please specify how I can apply this patch? Do I need to uninstall fastai (which I’ve installed using conda) first?

pellet · May 24, 2022, 11:26pm

Yeah uninstall fastai and pytorch.
Use the updated environment.yml to install the new conda environment and make sure it installs pytorch 1.13. After that you can build the patched fastai module by running pip install -e fastai

rawmean · May 25, 2022, 12:06am

Thanks. I still get the ‘fake _loader’ error. Do I need to specify any new options related to mps in the dataloader?

rawmean · May 25, 2022, 2:12am

I also had to add the highlighted line as line 29 to load.py.

But, now this is the problem. I guess we cannot use any of the existing model architectures until PyTorch implements them.
NotImplementedError: The operator 'aten::adaptive_max_pool2d.out' is not current implemented for the MPS device.

pellet · May 25, 2022, 4:28am

Ok, I thought I had merged the fix from fast.ai master which set pin_memory_device to an empty string. I didn’t have any trouble running the first part of the first fastbook notebook but haven’t tested it much… I didn’t see that error pop up for me, might need to fallback to cpu for that operation.

gerardo · May 25, 2022, 4:41pm

torch.backends.mps.is_available() is the equivalent to torch.cuda.is_available()

on this forum post @sgugger mentioned.

What is the equivalent when you have an Apple M1 processor?

learn = vision_learner(dls, resnet18, metrics=error_rate)
does not have device to be assigned

gerardo · May 25, 2022, 8:00pm

Honestly have anyone run successfully a test with fastai on Apple M1?
Can you share a notebook?

I found this notebook but does not include any fastai
You can see the GPU (Activity Monitor → Window → GPU history)

github.com

mrdbourke/pytorch-apple-silicon/blob/main/01_cifar10_tinyvgg.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f43f5b74",
   "metadata": {},
   "source": [
    "# PyTorch on Mac CIFAR10 TinyVGG Benchmark\n",
    "\n",
    "> **Note:** As of 21 May 2022, [PyTorch accelerated training for Mac](https://pytorch.org/blog/introducing-accelerated-pytorch-training-on-mac/) is in beta. And is only available through the nightly version of PyTorch.\n",
    "\n",
    "The following notebook tests the speed at which a given device can perform training iterations on the [CIFAR10 dataset](https://pytorch.org/vision/stable/generated/torchvision.datasets.CIFAR10.html) (10 classes, 50,000 training images, 10,000 testing images) training the [TinyVGG architecture from the CNN Explainer as a base](https://poloclub.github.io/cnn-explainer/).\n",
    "\n",
    "It's designed to be a simple test to compare PyTorch compute speeds on Apple Silicon CPU/GPUs.\n",
    "\n",
    "Ideally, PyTorch runs faster on Apple Silicon GPUs than on Apple Silicon CPUs.\n",
    "\n",
    "| Model | Dataset | Train Size | Test Size |\n",
    "| ----- | ----- | ----- | ----- |\n",
    "| [TinyVGG](https://poloclub.github.io/cnn-explainer/) (trained from scratch) | CIFAR10 (from `torchvision.datasets`) | ~50,000 (32x32x3) images | ~10,000 (32x32x3) images|\n",

This file has been truncated. show original

pellet · May 26, 2022, 1:27pm

Actually I fixed this error by chucking this in the top of the notebook i was using(before import torch)

#Crashes unless can fallback to cpu if an operation missing
os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"

pellet · May 26, 2022, 1:48pm

Yeah I got it running with some MPS support, you can see in this screenshot it took 4:30 to fine tune the resnet34 model in the first notebook 01_intro.ipynb :

Here is a link to my notebook: deep-learning-for-coders/01_intro.ipynb at master · pellet/deep-learning-for-coders · GitHub

mike.moloch · May 26, 2022, 2:11pm

Hi Ben, what are the specs of the M1 machine on which you got these results?

rawmean · May 26, 2022, 7:21pm

But with os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1", fast would run all the built-in architectures (eg, resnet34) on CPU, so it’s as if MPS does not exist and the nightly-build of PyTorch is and the changes that you have made to fastai are not needed.

I think we just need to wait for the PyTorch team to implement some of the layers that are used in all of the built-in architectures.