Fastai v3 with POWER(ppc64le architecture) machines

Hi, IBM cc labs through is giving a k80 with 20 cores and 32 Gig ram free of cost!. The catch is that the whole setup is on ppc64le architecture machine ,so normal wheels won’t work here, you need to build them from source.
Also one doesn’t have admin privileges , so you can’t do apt install package which is needed in some cases to install additional packages to build from source.
The workaround here is to use pre-installed conda environment and install fastai, pytorch and other ppc64le packages in it. I have successfully tested fastai 0.7 with pytorch 0.4.1 and it’s working with no issues. One additional library whose conda/wheel package for ppc64le arch I couldn’t find is py-arrow which is a dependency for feather-format.

Now for the fastai v1.0 library, I couldn’t find any prebuilt wheels nor for the pytorch-1.0dev. I would have built the wheels myself but I don’t have ppc64le arch machine.
If anyone could help here, then people can learn for free with a good setup.
Note There are sometimes no response issues and every time you login a docker container is spinned up to pull image, but that’s not an issue, since it just takes few seconds to install necessary packages. A script would automate this.


fastai is pure python. So if you can get pytorch v1 working, fastai will work too. Let us know if you get it working.

Can you build them on the IBM machine and upload from there? Best would be to build conda packages.

Yes jeremy. Just compiled fastai v1.0 from source for POWER. it compiled successfully.
However, the challenge is with the pytorch 1.0dev. @gsg is very generous to help me with this. Hopefully we will find a solution.

1 Like

unfortunately i dont have an IBM machine. The cloud IBM machine i am talking about doesn’t give root privileges, means couldn’t install necessary apt install [package] dependencies. Trying some workaround for this. Will keep you updated.

With conda you can install everything in a local folder, including c compilers, libs, binaries, etc.

1 Like

Got it. Will try and update accordingly. Thanks for the help.

Just got to know… CC labs might be AWS EC2 instances with k-80 :stuck_out_tongue:
I dont know how glances picked-up this information.