Thank you for reply. Is there any requirement.txt file for both fastai2 and fastcore Win 10?
I think it gets it from environment.yml file. Also, don’t forget to activate your environment from command line even if you use base environment. conda does not add it to path any more…
Mine is still not working, after I downgraded to Cuda 9. The environment.yml didn’t include Cuda. As I tried to install in different version of cuda or pytorch, it at the end uninstall something and then install something. End up it seems to be the same version of the uninstalled version. Thus, can you clarify which version to use. I notice that it is in the critical development stage between fastai1 and fastai2. I would appreciate if you can help me to clarify version that I needed for pytorch, torchvision and cuda. If you or the developers want me to try out different candidates, please give me a lists a may be able to try out different candidates.
I have been trying GCP, Win10, Ubuntu 19 Vbox in Win10, Gradient, Colab.
I was working with fastai ver1, which seem working fine for me. Then I thought there is no point to learned old version with new version coming out soon in a couple months and I need to learn the new version again.
Thanks.
my install seems using cuda10.1 with
pip install torch===1.4.0 torchvision===0.5.0 -f https://download.pytorch.org/whl/torch_stable.html
but I saw many other library remains of old install. I’s a mess right now 
Quick question? Are you having trouble with installing regular fastai2 or the editable install or both?
I have no trouble to install it and import fastai2. I think I had problem of training data, which is related to CUDA or GPU. I may have the setting wrong, but torch.cuda.is_avaiable == true, CUDA version now is 10.2.
good to hear all good now
Nice that you solved it. you should set number of workers for data loaders to 0. If not there you should add it. It cause many problems on windows. Also you need to install pip install --no-deps -e “.[dev]” I don’t install with dependencies… I update fastcore manually if needed.
hello, can you tell me how do you get working on a windows machine with num_workers set to 0? If I set the variable in the dataloader( I hope you are talking about dataloader inside torch/utils folder), I will get error at contractor of_MultiProcessingDataLoaderIter
super(_MultiProcessingDataLoaderIter, self).init(loader)
assert self._num_workers > 0
if loader.multiprocessing_context is None:
multiprocessing_context = multiprocessing
else:
multiprocessing_context = loader.multiprocessing_context
Can you help me? I got stuck in chapter1 and I just started this course. Can you provide information about the library version that got you running without error?
Thank you in advance.
Window or Not Window? Should this be a question?
What if your computer came with Window:
I think I tried hard with local Win10 in a notebook with
32G RAM, i7CPU, 1050Ti GPU, 250 SSD and 1T HD
Maybe I didn’t try hard enough.
Short answer:
I’ll try something else, rather than running fastai2 in Win10 locally. I may try Vbox with the new release of Ubuntu 20 in a few days.
Long answer, Here I shared my experience:
- Virtualbox or other virtual machines with Ubuntu??
It used to have issues with GPU drivers in Ubuntu 18 in Vbox, but it seems working better in Ubuntu 19. - Running Ubuntu 18(Window Sub Linux), it only allows me to install that in my SSD, which is almost full.
- Fastai2 running in the cloud is a general trend. Thus, I am moving to that direction with the Google Cloud Platform.
I would like to optimize my hardware locally, plus the advantage of cloud computing. Any recommendation?
Are you setting it in the way shown below (taken from https://github.com/fastai/fastai2/blob/master/nbs/course/lesson1-pets.ipynb)
dls = ImageDataLoaders.from_name_re(
path, fnames, pat=r'(.+)_\d+.jpg$', item_tfms=Resize(460), bs=bs,
batch_tfms=[*aug_transforms(size=224, min_scale=0.75), Normalize.from_stats(*imagenet_stats)],num_workers=0)
as this works for me on windows?
On a separate note I am unable to get this to work with the nlp data loader(https://github.com/fastai/fastai2/blob/master/nbs/course/lesson3-imdb.ipynb). That is setting
imdb_lm = DataBlock(blocks=(TextBlock.from_df('text', is_lm=True,n_workers=0),),
get_x=ColReader('text'),
splitter=RandomSplitter())
results in the following error
~\Anaconda3\envs\fastai2\lib\site-packages\numpy\lib\shape_base.py in array_split(ary, indices_or_sections, axis)
774 Nsections = int(indices_or_sections)
775 if Nsections <= 0:
–> 776 raise ValueError(‘number sections must be larger than 0.’)
777 Neach_section, extras = divmod(Ntotal, Nsections)
778 section_sizes = ([0] +ValueError: number sections must be larger than 0.
dls = ImageDataLoaders.from_name_re(
path, fnames, pat=r'(.+)_\d+.jpg$', item_tfms=Resize(460), bs=bs,
batch_tfms=[*aug_transforms(size=224, min_scale=0.75), Normalize.from_stats(*imagenet_stats)],num_workers=0)
This worked for me. Thank you very much. 
Has anyone managed to successfully run through lesson-3-imdb (https://github.com/fastai/fastai2/blob/master/nbs/course/lesson3-imdb.ipynb) on windows? I have tried to limit the number of workers in the data loader to 0 in a similar (n_workers instead of num_workers) way as the image data loader however on both linux and windows on the latest version of fastai2 (0.0.16) when I try to use the data loader with n_workers=0 as below
imdb_lm = DataBlock(blocks=(TextBlock.from_df('text', is_lm=True,n_workers=0),),
get_x=ColReader('text'),
splitter=RandomSplitter())
I get the following error
ValueError: number sections must be larger than 0.
TypeError Traceback (most recent call last)
~/anaconda3/envs/fastai2/lib/python3.7/site-packages/numpy/lib/shape_base.py in array_split(ary, indices_or_sections, axis)
769 # handle array case.
–> 770 Nsections = len(indices_or_sections) + 1
771 div_points = [0] + list(indices_or_sections) + [Ntotal]TypeError: object of type ‘int’ has no len()
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
in
3 splitter=RandomSplitter())
4
----> 5 dbunch_lm = imdb_lm.dataloaders(df)~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/data/block.py in dataloaders(self, source, path, verbose, **kwargs)
105
106 def dataloaders(self, source, path=’.’, verbose=False, **kwargs):
–> 107 dsets = self.datasets(source)
108 kwargs = {**self.dls_kwargs, **kwargs, ‘verbose’: verbose}
109 return dsets.dataloaders(path=path, after_item=self.item_tfms, after_batch=self.batch_tfms, **kwargs)~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/data/block.py in datasets(self, source, verbose)
102 splits = (self.splitter or RandomSplitter())(items)
103 pv(f"{len(splits)} datasets of sizes {’,’.join([str(len(s)) for s in splits])}", verbose)
–> 104 return Datasets(items, tfms=self._combine_type_tfms(), splits=splits, dl_type=self.dl_type, n_inp=self.n_inp, verbose=verbose)
105
106 def dataloaders(self, source, path=’.’, verbose=False, **kwargs):~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/data/core.py in init(self, items, tfms, tls, n_inp, dl_type, **kwargs)
278 def init(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
279 super().init(dl_type=dl_type)
–> 280 self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
281 self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
282~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/data/core.py in (.0)
278 def init(self, items=None, tfms=None, tls=None, n_inp=None, dl_type=None, **kwargs):
279 super().init(dl_type=dl_type)
–> 280 self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
281 self.n_inp = ifnone(n_inp, max(1, len(self.tls)-1))
282~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/foundation.py in call(cls, x, args, **kwargs)
39 return x
40
—> 41 res = super().call(((x,) + args), **kwargs)
42 res._newchk = 0
43 return res~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/data/core.py in init(self, items, tfms, use_list, do_setup, split_idx, train_setup, splits, types, verbose)
216 if do_setup:
217 pv(f"Setting up {self.tfms}", verbose)
–> 218 self.setup(train_setup=train_setup)
219
220 def _new(self, items, split_idx=None, **kwargs):~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/data/core.py in setup(self, train_setup)
232
233 def setup(self, train_setup=True):
–> 234 self.tfms.setup(self, train_setup)
235 if len(self) != 0:
236 x = super().getitem(0) if self.splits is None else super().getitem(self.splits[0])[0]~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/transform.py in setup(self, items, train_setup)
177 tfms = self.fs[:]
178 self.fs.clear()
–> 179 for t in tfms: self.add(t,items, train_setup)
180
181 def add(self,t, items=None, train_setup=False):~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/transform.py in add(self, t, items, train_setup)
180
181 def add(self,t, items=None, train_setup=False):
–> 182 t.setup(items, train_setup)
183 self.fs.append(t)
184~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/transform.py in setup(self, items, train_setup)
76 def setup(self, items=None, train_setup=False):
77 train_setup = train_setup if self.train_setup is None else self.train_setup
—> 78 return self.setups(getattr(items, ‘train’, items) if train_setup else items)
79
80 def _call(self, fn, x, split_idx=None, **kwargs):~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/dispatch.py in call(self, *args, **kwargs)
96 if not f: return args[0]
97 if self.inst is not None: f = MethodType(f, self.inst)
—> 98 return f(*args, **kwargs)
99
100 def get(self, inst, owner):~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/text/core.py in setups(self, dsets)
285 def setups(self, dsets):
286 if not self.mode == ‘df’ or not isinstance(dsets.items, pd.DataFrame): return
–> 287 dsets.items,count = tokenize_df(dsets.items, self.text_cols, rules=self.rules, **self.kwargs)
288 if self.counter is None: self.counter = count
289 return dsets~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/text/core.py in tokenize_df(df, text_cols, n_workers, rules, mark_fields, tok_func, res_col_name, **tok_kwargs)
215 rules = L(ifnone(rules, defaults.text_proc_rules.copy()))
216 texts = _join_texts(df[text_cols], mark_fields=mark_fields)
–> 217 outputs = L(parallel_tokenize(texts, tok_func, rules, n_workers=n_workers, **tok_kwargs)
218 ).sorted().itemgot(1)
219~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/foundation.py in call(cls, x, args, **kwargs)
39 return x
40
—> 41 res = super().call(((x,) + args), **kwargs)
42 res._newchk = 0
43 return res~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/foundation.py in init(self, items, use_list, match, *rest)
312 if items is None: items = []
313 if (use_list is not None) or not _is_array(items):
–> 314 items = list(items) if use_list else _listify(items)
315 if match is not None:
316 if is_coll(match): match = len(match)~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastcore/foundation.py in _listify(o)
248 if isinstance(o, list): return o
249 if isinstance(o, str) or _is_array(o): return [o]
–> 250 if is_iter(o): return list(o)
251 return [o]
252~/anaconda3/envs/fastai2/lib/python3.7/site-packages/fastai2/torch_core.py in parallel_gen(cls, items, n_workers, as_gen, **kwargs)
719 def parallel_gen(cls, items, n_workers=defaults.cpus, as_gen=False, **kwargs):
720 “Instantiateclsinn_workersprocs & call each on a subset ofitemsin parallel.”
–> 721 batches = np.array_split(items, n_workers)
722 idx = np.cumsum(0 + L(batches).map(len))
723 queue = Queue()<array_function internals> in array_split(*args, **kwargs)
~/anaconda3/envs/fastai2/lib/python3.7/site-packages/numpy/lib/shape_base.py in array_split(ary, indices_or_sections, axis)
774 Nsections = int(indices_or_sections)
775 if Nsections <= 0:
–> 776 raise ValueError(‘number sections must be larger than 0.’)
777 Neach_section, extras = divmod(Ntotal, Nsections)
778 section_sizes = ([0] +ValueError: number sections must be larger than 0.
I just bonked my head on this today on a gcp instance in which fastai2 came preloaded.
Is there any reason pip install --upgrade fastai2 shouldn’t update fastcore based on the dependency checks? (It didn’t.)
We’re still in pre-release mode, the dependency with fastcore is not vesion-tagged (cause we would need to increase it all the time) and we always say in all our instructions to update fastai2 and fastcore together.
2 Likes
Hello, I am not able to open a new topic. Therefore I thought it would be the best option to post my problem here as I am using a windows platform.
I am getting an error msg in python running this code (which is the begining example code in fastai lessons) below, … after installing fastai2 and updating fastcore with pip install fastai2 and pip install fastcore
the code:
from utils import *
from fastai2.vision.all import *
path = untar_data(URLs.PETS)/‘images’
def is_cat(x): return x[0].isupper()
dls = ImageDataLoaders.from_name_func(path, get_image_files(path), valid_pct=0.2, seed=42, label_func=is_cat, item_tfms=Resize(224))
learn = cnn_learner(dls, resnet34, metrics=error_rate)
learn.fine_tune(1)
the error msg:
c:\users\user\appdata\local\programs\python\python38\lib\site-packages\torch\cuda_init_.py:52: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 9010). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at …\c10\cuda\CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0
Traceback (most recent call last):
File “C:\Users\user\Desktop\fastai\fastai1.py”, line 5, in
from fastai2.vision.all import *
File “c:\users\user\fastai2\fastai2\vision\all.py”, line 1, in
from …basics import *
File “c:\users\user\fastai2\fastai2\basics.py”, line 1, in
from .data.all import *
File “c:\users\user\fastai2\fastai2\data\all.py”, line 1, in
from …torch_basics import *
File “c:\users\user\fastai2\fastai2\torch_basics.py”, line 5, in
from .layers import *
File “c:\users\user\fastai2\fastai2\layers.py”, line 276, in
@log_args
NameError: name ‘log_args’ is not defined
I have also tried below methods as it is mentioned somewhere in forums. But again the same error msg shows.
git clone https://github.com/fastai/fastai2
cd fastai2
pip install -e ".[dev]"
which installs fastai 0.0.30 version as a result
git clone https://github.com/fastai/fastcore/
cd fastcore
pip install -e ".[dev]"
I would appreciate if you could tell me how to solve this problem.
@Linda, fastai2 is old name and absolute now.
you can check https://github.com/fastai/fastai
try
git clone https://github.com/fastai/fastai
pip install -e “fastai[dev]”
pip install -e “fastai[dev]” from the fastai repo.
not fastai2.
dls = ImageDataLoaders.from_name_func(path, get_image_files(path), valid_pct=0.2, seed=42, label_func=is_cat, item_tfms=Resize(224), num_workers=0)
num_workers=0 has to set 0 for windows. Also you should import fastai not fastai2.
I am happy to inform you that, after making the changes which you have adviced, I have finally got some results.
In the meanwhile, before the results, python gave me some warnings. From the warnings it is understood that I shall update NVIDIA driver… I will check if this is possible with my hardware… After the text there are some other things written in programmatic expressions. I am wondering if I should take any steps to correct anything? Or are they just some explanatory things about the oldish NVIDIA driver? Many thanks.
This is the complete output:
c:\users\user\appdata\local\programs\python\python38\lib\site-packages\torch\cuda_init_.py:52: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 9010). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at …\c10\cuda\CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0
c:\users\user\fastai\fastai\callback\hook.py:190: SyntaxWarning: “is not” with a literal. Did you mean “!=”?epoch train_loss valid_loss error_rate time
0 0.164075 0.022414 0.009472 32:07
epoch train_loss valid_loss error_rate time
0 0.048312 0.010725 0.002706 39:42
Good to hear it’s working.
As you noticed you should check and update your NVIDIA system drivers. Than try to install cuda driver to one of 10.1, 10.2 or 11 it depends on your hardware. As a last step install pytorch with right cuda support e.g. 10.1 version.
Hi! I am facing the same error when creating a dataloader. Have you been able to solve this error?
Blockquote
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py”, line 116, in spawn_main
exitcode = _main(fd, parent_sentinel)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py”, line 125, in _main
prepare(preparation_data)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py”, line 236, in prepare
_fixup_main_from_path(data[‘init_main_from_path’])
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py”, line 287, in _fixup_main_from_path
main_content = runpy.run_path(main_path,
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\runpy.py”, line 265, in run_path
return _run_module_code(code, init_globals, run_name,
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\runpy.py”, line 97, in _run_module_code
_run_code(code, mod_globals, init_globals,
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\runpy.py”, line 87, in _run_code
exec(code, run_globals)
File “C:\Users\Morpheus\Documents\Cyber-Dive-App\backend\mia\alerts\alerts_fastaiv2_hf.py”, line 420, in
alertsfinal = alerts_roberta(alerts_df)
File “C:\Users\Morpheus\Documents\Cyber-Dive-App\backend\mia\alerts\alerts_fastaiv2_hf.py”, line 324, in alerts_roberta
dsets = Datasets(df, splits=splits, tfms=[
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastai2\data\core.py”, line 289, in init
self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastai2\data\core.py”, line 289, in
self.tls = L(tls if tls else [TfmdLists(items, t, **kwargs) for t in L(ifnone(tfms,[None]))])
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\foundation.py”, line 47, in call
res = super().call(*((x,) + args), **kwargs)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastai2\data\core.py”, line 226, in init
self.setup(train_setup=train_setup)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastai2\data\core.py”, line 242, in setup
self.tfms.setup(self, train_setup)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\transform.py”, line 197, in setup
for t in tfms: self.add(t,items, train_setup)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\transform.py”, line 200, in add
t.setup(items, train_setup)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\transform.py”, line 78, in setup
return self.setups(getattr(items, ‘train’, items) if train_setup else items)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\dispatch.py”, line 99, in call
return f(args, **kwargs)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastai2\text\core.py”, line 285, in setups
dsets.items,count = tokenize_df(dsets.items, self.text_cols, rules=self.rules, **self.kwargs)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastai2\text\core.py”, line 219, in tokenize_df
outputs = L(parallel_tokenize(texts, tok, rules, n_workers=n_workers)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\foundation.py”, line 47, in call
res = super().call(((x,) + args), **kwargs)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\foundation.py”, line 324, in init
items = list(items) if use_list else _listify(items)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\foundation.py”, line 260, in _listify
if is_iter(o): return list(o)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\utils.py”, line 760, in parallel_gen
yield from run_procs(f, done, L(batches,idx).zip())
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\site-packages\fastcore\utils.py”, line 738, in run_procs
for o in processes: o.start()
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\process.py”, line 121, in start
self._popen = self._Popen(self)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py”, line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\context.py”, line 327, in _Popen
return Popen(process_obj)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\popen_spawn_win32.py”, line 45, in init
prep_data = spawn.get_preparation_data(process_obj._name)
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py”, line 154, in get_preparation_data
_check_not_importing_main()
File “C:\Users\Morpheus\AppData\Local\Programs\Python\Python38\lib\multiprocessing\spawn.py”, line 134, in _check_not_importing_main
raise RuntimeError(’’’
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.
This probably means that you are not using fork to start your
child processes and you have forgotten to use the proper idiom
in the main module:
if __name__ == '__main__':
freeze_support()
...
The "freeze_support()" line can be omitted if the program
is not going to be frozen to produce an executable.