Developer chat


(Jeremy Howard) #102

Working on it:

http://docs.fast.ai/gen_doc.html


(Stas Bekman) #103

Hmm, but now we no longer have .md to work with. there is no way to preserve the easy to edit format?

we have developers.md at the moment, we can’t have the same for doc gen and everything else?

Also most of the instructions that got moved from CONTRIBUTING aren’t belonging to doc-gen, should be in a different file.


(Stas Bekman) #104

heads up to a few renames under tools/:

git mv tools/trust-doc-nbs tools/trust-doc-nbs-install-hook
git mv tools/trust-doc-nbs-hook tools/trust-doc-nbs

but it should be transparent to you if you use tools/run-after-git-clone.

and another rename:

git mv tools/build tools/sync-nb-exports

so now it’s easier to know what we are trying to do.


(Jeremy Howard) #105

We’re using our documentation framework to document our documentation framework. :slight_smile:


#106

In case you missed it, fastai_pytorch is now merged on the fastai main repo. PRs should now be done there and only then. The old fastai library is in the old folder, the docs sources have moved in this repo.


Fastai_v1 repo changed to fastai_pytorch
(Yashu Seth) #107

All the Source links (that point to the source code) in the documentation results in a 404. Could be due to these recent changes (fastai_pytorch is now merged on the fastai main repo).


(Kaspar Lund) #108

So congrates with the major upgrade of fastai.

Will you incorporate the the brilliant notebooks like 001a_nn_basics.ipynb later or will they stay in “fastai old”


(Jeremy Howard) #109

001a will become an official pytorch tutorial on their site. The remaining notebooks will form the basis of the part2 course next year.


(Kaspar Lund) #110

I believe the adresses on https://anaconda.org/fastai/fastai are point on the old repos ?


(Jeremy Howard) #111

Thanks for the reminder :slight_smile:


(Stas Bekman) #112

Thank you, @Kaspar, will be fixed in the next upload.

update: done


#113

I think they’re all back working now. Thanks for pointing this out.


(Arka Sadhu) #114

EDIT: I solved it. I had to first run conda install conda. After that it worked flawlessly.

Leaving the rest of it verbatim, if someone also runs into the same error.

I am not sure why, but I am unable to install fastai on a new environment. I am trying conda install -c pytorch -c fastai fastai pytorch-nightly cuda90 (since I have cuda9.0). However, I am getting the following error:

Output I am getting:
Solving environment: failed

PackagesNotFoundError: The following packages are not available from current channels:

  • fastai
  • fastprogress

Current channels:

To search for alternate channels that may provide the conda package you’re
looking for, navigate to

https://anaconda.org

and use the search bar at the top of the page.

I was able to install the fastai_v1 repository earlier via pip install -e .

Can anyone confirm the error? the anaconda repo seems correctly linked, so not sure why there is an error.


(Stas Bekman) #115

Thank you for the report, @TheShadow29.

Can you please elaborate so that we know whether this is something that needs to be documented? How could you possibly have conda environment w/o conda installed? I’m not sure how you could run conda to install conda. Did you perhaps mean conda update conda?

Hmm, I have just tried conda uninstall conda and it actually allows that. I didn’t proceed, I am not sure why this is even possible. There must be a default thin conda util that gets installed by default. Too bad you didn’t catch --version before installing conda.

I think the reason the unupdated conda wasn’t finding the packages, is because -c fastai is not really a channel, but just a user on anaconda.org, they might have made it possible to use user’s account as a channel somewhat recently (it is still not fully supported by conda-build).


(Marco Ribeiro) #116

I’m having a hard time creating a fastai_v1 enviroment. Here is what I’m doing:

conda create --name fastai_v1
conda activate fastai_v1
conda install -c pytorch -c fastai fastai pytorch-nightly cuda92
conda uninstall fastai
pip install -e . ##inside the fastai repository folder

After that, when I try to run a notebook of the examples folder, a get this error:

ModuleNotFoundError: No module named 'torch' 

I can import the torch package in the python terminal:

>>> import torch; torch.__version__
'1.0.0.dev20180930'

Should I do something different?

What is the environment.yml file in the fastai root folder for in this new version?


(Stas Bekman) #117

heads up: please don’t make ‘import fastai’ rely on any of the jupyter stuff, as it breaks conda-build - none of the jupyter stuff is on conda - it be a monumental project to put all the dependencies there.
So I had to remove this:

fastai/__init__.py:
- from .gen_doc.nbdoc import nbshow

(Stas Bekman) #118

I’m trying to get the environment tested w/o needing to run the notebook.

This worked:

jupyter nbconvert --execute --ExecutePreprocessor.timeout=600 --to notebook examples/tabular.ipynb

This fails:

jupyter nbconvert --execute --ExecutePreprocessor.timeout=600 --to notebook examples/cifar.ipynb

[NbConvertApp] WARNING | Config option `template_path` not recognized by `NotebookExporter`.
[NbConvertApp] Converting notebook examples/cifar.ipynb to notebook
[NbConvertApp] Executing notebook with kernel: python3
[NbConvertApp] ERROR | Error while converting 'examples/cifar.ipynb'
Traceback (most recent call last):
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/nbconvertapp.py", line 393, in export_single_notebook
    output, resources = self.exporter.from_filename(notebook_filename, resources=resources)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 174, in from_filename
    return self.from_file(f, resources=resources, **kw)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 192, in from_file
    return self.from_notebook_node(nbformat.read(file_stream, as_version=4), resources=resources, **kw)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/exporters/notebook.py", line 31, in from_notebook_node
    nb_copy, resources = super(NotebookExporter, self).from_notebook_node(nb, resources, **kw)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 134, in from_notebook_node
    nb_copy, resources = self._preprocess(nb_copy, resources)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/exporters/exporter.py", line 311, in _preprocess
    nbc, resc = preprocessor(nbc, resc)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 47, in __call__
    return self.preprocess(nb, resources)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 262, in preprocess
    nb, resources = super(ExecutePreprocessor, self).preprocess(nb, resources)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/preprocessors/base.py", line 69, in preprocess
    nb.cells[index], resources = self.preprocess_cell(cell, resources, index)
  File "/home/stas/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/nbconvert/preprocessors/execute.py", line 286, in preprocess_cell
    raise CellExecutionError.from_cell_and_msg(cell, out)
nbconvert.preprocessors.execute.CellExecutionError: An error occurred while executing the following cell:
------------------
untar_data(CIFAR_PATH)
------------------

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-2-aac8ec0ed7ac> in <module>()
----> 1 untar_data(CIFAR_PATH)

/mnt/disc1/fast.ai-1/br/fastai/master/fastai/docs.py in untar_data(path)
     25
     26 def untar_data(path):
---> 27     download_data(path.name)
     28     if not path.exists(): tarfile.open(f_name(path), 'r:gz').extractall(DATA_PATH)
     29

/mnt/disc1/fast.ai-1/br/fastai/master/fastai/docs.py in download_data(name)
     22 def download_data(name):
     23     dest = DATA_PATH/f_name(name)
---> 24     if not dest.exists(): download_url(f'{URL}{f_name(name)}', dest)
     25
     26 def untar_data(path):

/mnt/disc1/fast.ai-1/br/fastai/master/fastai/core.py in download_url(url, dest, overwrite)
    153
    154     with open(dest,'wb') as f:
--> 155         pbar = progress_bar(range(file_size), auto_update=False)
    156         nbytes,buffer = 0,[1]
    157         while len(buffer):

TypeError: __init__() got an unexpected keyword argument 'auto_update'
TypeError: __init__() got an unexpected keyword argument 'auto_update'

update: it breaks in jupyter too, the same way.

It’d probably make a good basic test to just successfully run the example notebooks from the command line.


(Jeremy Howard) #119

That’s an out of date fastprogress version.


(Stas Bekman) #120

@elmarculino, I have just followed your steps plus:

pip install jupyter_contrib_nbextensions ipywidgets

and loading pytorch works for me from the jupyter notebook.

Perhaps print your sys.path and see whether you’re using the wrong environment when you run jupyter notebook?


(Marco Ribeiro) #121

You are right. The path was wrong.

From notebook, using jupyter lab, i got the error ModuleNotFoundError: No module named ‘torch’:

['',
 '/home/marco/anaconda3/lib/python36.zip',
 '/home/marco/anaconda3/lib/python3.6',
 '/home/marco/anaconda3/lib/python3.6/lib-dynload',
 '/home/marco/anaconda3/lib/python3.6/site-packages',
 '/mnt/ssd32gb/Arquivos/MachineLearning/fastai_pytorch',
 '/home/marco/anaconda3/lib/python3.6/site-packages/IPython/extensions',
 '/home/marco/.ipython']

From notebook, using jupyter notebook:

['',
 '/mnt/ssd32gb/Arquivos/MachineLearning/fastai-repo/examples',
 '/home/marco/anaconda3/envs/fastai_v1/lib/python37.zip',
 '/home/marco/anaconda3/envs/fastai_v1/lib/python3.7',
 '/home/marco/anaconda3/envs/fastai_v1/lib/python3.7/lib-dynload',
 '/home/marco/anaconda3/envs/fastai_v1/lib/python3.7/site-packages',
 '/mnt/ssd32gb/Arquivos/MachineLearning/fastai-repo',
 '/home/marco/anaconda3/envs/fastai_v1/lib/python3.7/site-packages/IPython/extensions',
 '/home/marco/.ipython']

Here the kernel always restarts:

[W 01:19:16.595 NotebookApp] Notebook fastai-repo/examples/cifar.ipynb is not trusted
[I 01:19:17.113 NotebookApp] Kernel started: ed9289cd-5b3c-4acc-86fd-5e702248e09e
[I 01:19:17.681 NotebookApp] Adapting to protocol v5.1 for kernel ed9289cd-5b3c-4acc-86fd-5e702248e09e
[I 01:19:32.109 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
WARNING:root:kernel ed9289cd-5b3c-4acc-86fd-5e702248e09e restarted
[I 01:20:05.127 NotebookApp] KernelRestarter: restarting kernel (1/5), keep random ports
WARNING:root:kernel ed9289cd-5b3c-4acc-86fd-5e702248e09e restarted

Python console:

>>> sys.path
['', 
'/home/marco/anaconda3/envs/fastai_v1/lib/python37.zip', 
'/home/marco/anaconda3/envs/fastai_v1/lib/python3.7', 
'/home/marco/anaconda3/envs/fastai_v1/lib/python3.7/lib-dynload', 
'/home/marco/anaconda3/envs/fastai_v1/lib/python3.7/site-packages', 
'/mnt/ssd32gb/Arquivos/MachineLearning/fastai-repo']

I will reinstall my anaconda and try again. Thanks for your help!