Fastai v2 chat

arora_aman · November 7, 2019, 10:43pm

Never mind for now

Different dsrc objects passed to Categorize lead to this inconsistency.

I will keep looking and see why that happens.

# tl dsrc
TfmdList: [PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/basset_hound_111.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/Siamese_178.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/keeshond_34.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/german_shorthaired_94.jpg')]
tfms - (#1) [Transform: True (object,object) -> RegexLabeller ]


# pipe dsrc
TfmdList: [PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/basset_hound_111.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/Siamese_178.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/keeshond_34.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/german_shorthaired_94.jpg')]
tfms - (#2) [Transform: True (object,object) -> RegexLabeller ,Categorize: True (object,object) -> encodes (object,object) -> decodes]

jeremy · November 7, 2019, 11:53pm

bcolz is a great choice. You can also create iterable datasets in fastai v2. Have a look at the notebook where DataLoader is defined to see various approaches to that. Since I haven’t actually had a need to use them myself yet, they’re not tested in a practical setting - so do let us know if you try them out and have any issues.

It’s a pleasure!

arora_aman · November 8, 2019, 4:39am

Somehow, I haven’t been able to replicate setup method of TfmdList, could I please double check my understanding of the code?

Referring to the image:
The first step is to getattr(self, 'train', self) which means, we get the train subset where i=0 and therefore, in step-2, we get splits[0], and return items with that split.

Then, in step-2 itself, we call _new which takes us to step-4 (skipping step-3 as that is only _get to get items defined in L), where we call super._new(items, tfms=self.tfms, do_setup=False, **kwargs).

This takes us to step-5, because _new is defined inside L which all it does is return a new object of type TfmdList with items with idx in split[0], do_setup=False. So we get a new TfmdList with same tfms.

This takes us back to step-6, because now we have self.train which is a TfmdList with items in splits[0].
In step-6., we call self.tfms.setup() passing in this very self.train object.

Since self.tfms is a Pipeline, now we get to step-7 which finally performs setup on the individual Transforms which for pets tutorial for y variable are [RegexLabeller(pat), Categorize].

This Pipeline object self is nothing but TfmdList.tfms or TfmdList.train.tfms, right? The items arg that we have passed to step-7, is the train TfmdList with the same tfms as the original TfmdList.

Therefore, this items obj, (which is nothing but original TfmdLists train subset) passed to Pipeline.setup looks something like:

TfmdList: [PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/basset_hound_111.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/Siamese_178.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/keeshond_34.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/german_shorthaired_94.jpg')]
tfms - (#2) [Transform: True (object,object) -> RegexLabeller ,Categorize: True (object,object) -> encodes (object,object) -> decodes]

If originally,

items
>> (#5) [/home/ubuntu/.fastai/data/oxford-iiit-pet/images/keeshond_34.jpg,/home/ubuntu/.fastai/data/oxford-iiit-pet/images/Siamese_178.jpg,/home/ubuntu/.fastai/data/oxford-iiit-pet/images/german_shorthaired_94.jpg,/home/ubuntu/.fastai/data/oxford-iiit-pet/images/Abyssinian_92.jpg,/home/ubuntu/.fastai/data/oxford-iiit-pet/images/basset_hound_111.jpg]

splits
>> ((#4) [4,1,0,2], (#1) [3])

tfms
>> [<local.data.transforms.RegexLabeller at 0x7f49c026fd30>,
 local.data.transforms.Categorize]

This is where it gets a little confusing.

When we do self.fs.clear(), the items are now:

ipdb> items
TfmdList: [PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/basset_hound_111.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/Siamese_178.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/keeshond_34.jpg'), PosixPath('/home/ubuntu/.fastai/data/oxford-iiit-pet/images/german_shorthaired_94.jpg')]
tfms - (#0) []

The tfms are cleared on items which was the train subset of original TfmdList!!

Why would this be?

My understanding:

A TfmdList and a TfmdList.subset(0) share the same tfms stored in both objects self.tfms
This self.tfms is a Pipeline that we call setup on, thus when we do self.fs.clear(), it clears the Transforms on TfmdList and TfmdList.subset(0).
Since this TfmdList.subset(0) is what get’s passed as items, therefore, tfms get cleared when we do self.fs.clear().

Following on the above understanding, therefore, when I do:

_tl = TfmdList(items[splits[0]], tfms=[RegexLabeller(pat)], do_setup=False)
pipe = Pipeline(tfms, as_item=True); 
pipe.setup(_tl)
pipe.vocab

>> (#4) [Siamese,basset_hound,german_shorthaired,keeshond]

It works

foobar8675 · November 11, 2019, 2:39am

can I get some advice with installing v2? On my windows 10 machine I’m doing :

git clone git@github.com:fastai/fastai_dev.git
cd .\fastai_dev\
conda env create -f environment.yml
conda activate fastai_dev
pip install git+https://github.com/fastai/fastai_dev

then running python from the shell where I get the following error

>>> from fastai2.basics import *
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'fastai2'

Does anyone have any advice?

muellerzr · November 12, 2019, 2:39am

I had a thought and I want your opinion before I try to do anything large like this. We have ClassificationInterpretation, but why not just an Interpreter. Don’t limit to classification. For example, regression. We plot the worst guesses (for image points), or for tabular we do a cluster of how close the guesses were to the correct point. Also we could implement permutation importance in for tabular here too, to make it easier to analyze tabular models. In terms of NLP, I need to revisit the NLP course again but I imagine something along the lines of word clustering and how close we came for our Language Models. I can try and come up with something for the tabular or vision models if you want a visualization (which I know always helps).

Let me know your thoughts on this and if you know of any other regression-based analysis techniques that should be included in the above.

ilovescience · November 12, 2019, 4:54am

Thought I might mention that there are SegmentationInterpretation and TextClassificationInterpretation classes already exist. I am not sure if it makes sense to have a general Interpreter class as each task will have different interpreting methods, so I think that’s why current approach has been to have separate classes for separate tasks.

muellerzr · November 12, 2019, 4:57am

I suppose that makes a lot more sense and sounds better I was thinking something along the lines (in terms of code) a generic that can pick up what type is being passed in and apply particular functions to it, hence the Interpreter class.

ilovescience · November 12, 2019, 5:00am

Sure I guess it wouldn’t be hard to do something like that with type dispatching (IIRC that’s the correct term). However, it’s still hard as a user because there aren’t any set functions because there are different interpretation functions for each task.

muellerzr · November 12, 2019, 5:03am

Thanks for the check! (We just learned about type dispatching in my Intermidate class a few weeks ago, forgot the proper term for a moment).

That makes sense, so perhaps then mabye TextInterpretation, VisionInterpretation, and TabularInterpretation? (If these regression/language inferences were added)

ilovescience · November 12, 2019, 5:09am

Again, there could be various NLP or vision or tabular tasks with their own interpretation methods. However, I don’t want to be too negative and I will let Jeremy and Sylvain decide what is the best approach over here.

muellerzr · November 12, 2019, 5:11am

You’re certainly not being too negative Constructed criticism is always welcome For now, I’ll wait for Jeremy or Sylvain, and lull over which specific implementation ideas could convert over to one of the existing interps easily (a feature importance or feature visualization could easily be done IMO for tabular atleast)

sgugger · November 12, 2019, 2:43pm

There is already an interpretation class, and a plot_top_losses method that has a type-dispatched counterpart (like show_batch and show_results). For now it only handles image and text classification but more visualization cases can be added with the type-dispatch.

muellerzr · November 12, 2019, 3:01pm

Got it! I’ll look into that more. Thanks @sgugger

arora_aman · November 14, 2019, 12:40am

Just a very quick code question for anyone please:

 dls = [dl_type(self.subset(i), bs=b, shuffle=s, drop_last=s, n=n if i==0 else None, **kwargs, **dk)
                for i,(b,s,dk) in enumerate(zip(bss,shuffles,dl_kwargs))]

Inside, FilteredBase databunch method, regarding above code, does this mean that we pass None to dl_type when i != 0 ?

Seems like we pass None:

[(i, b) if i==0 else None for i,b in enumerate([1,5])]

>> [(0, 1), None]

Follow up question then, how does the second dl get it’s dataset then? :o

arora_aman · November 14, 2019, 5:02am

My bad

The if-else condition is only for n=n if i==0 else, I wasn’t aware we could have if else inside function args too.

Something new to learn everyday

ulat · November 17, 2019, 6:46pm

I am bit confused about swift4tensorflow. Will fastai v2 be plain python or shifted to swift/tf ?

ilovescience · November 17, 2019, 7:41pm

it’s in python. S4TF is a separate project.

jeremy · November 17, 2019, 10:07pm

This is called the “ternary operator”: https://book.pythontips.com/en/latest/ternary_operators.html

muellerzr · November 18, 2019, 7:16pm

I know this has been addressed earlier, but today I suddenly started getting NBMasterBar errors doing a fit_one_cycle.

AttributeError: 'NBMasterBar' object has no attribute 'out'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
AttributeError: pbar

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in on_iter_end(self)
    199             plt.close()
    200             self.out2.update(self.fig)
--> 201         total_time = format_time(time() - self.start_t)
    202         if self.text.endswith('<p>'): self.text = self.text[:-3]
    203         if self.total_time: self.text = f'Total time: {total_time} <p>' + self.text

AttributeError: 'NBMasterBar' object has no attribute 'start_t'

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in _do_epoch_train(self)
    254         try:
--> 255             self.dl = self.dbunch.train_dl;                  self('begin_train')
    256             self.all_batches()

44 frames
/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in __call__(self, event_name)
    214 
--> 215     def __call__(self, event_name): L(event_name).map(self._call_one)
    216     def _call_one(self, event_name):

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in map(self, f, *args, **kwargs)
    362              else f.__getitem__)
--> 363         return self._new(map(g, self))
    364 

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in _new(self, items, *args, **kwargs)
    315     def _xtra(self): return None
--> 316     def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)
    317     def __getitem__(self, idx): return self._get(idx) if is_indexer(idx) else L(self._get(idx), use_list=None)

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __call__(cls, x, *args, **kwargs)
     41 
---> 42         res = super().__call__(*((x,) + args), **kwargs)
     43         res._newchk = 0

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __init__(self, items, use_list, match, *rest)
    306         if (use_list is not None) or not _is_array(items):
--> 307             items = list(items) if use_list else _listify(items)
    308         if match is not None:

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in _listify(o)
    242     if isinstance(o, str) or _is_array(o): return [o]
--> 243     if is_iter(o): return list(o)
    244     return [o]

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __call__(self, *args, **kwargs)
    208         fargs = [args[x.i] if isinstance(x, _Arg) else x for x in self.pargs] + args[self.maxi+1:]
--> 209         return self.fn(*fargs, **kwargs)
    210 

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in _call_one(self, event_name)
    217         assert hasattr(event, event_name)
--> 218         [cb(event_name) for cb in sort_by_run(self.cbs)]
    219 

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in <listcomp>(.0)
    217         assert hasattr(event, event_name)
--> 218         [cb(event_name) for cb in sort_by_run(self.cbs)]
    219 

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in __call__(self, event_name)
     20         "Call `self.{event_name}` if it's defined"
---> 21         if self.run: getattr(self, event_name, noop)()
     22 

/usr/local/lib/python3.6/dist-packages/fastai2/callback/progress.py in begin_train(self)
     25 
---> 26     def begin_train(self):    self._launch_pbar()
     27     def begin_validate(self): self._launch_pbar()

/usr/local/lib/python3.6/dist-packages/fastai2/callback/progress.py in _launch_pbar(self)
     34     def _launch_pbar(self):
---> 35         self.pbar = progress_bar(self.dl, parent=getattr(self, 'mbar', None), leave=False)
     36         if self.pbar.parent is None: self.pbar.on_iter_begin()

/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in __init__(self, gen, total, display, leave, parent, auto_update)
    158         self.progress = html_progress_bar(0, len(gen) if total is None else total, "")
--> 159         super().__init__(gen, total, display, leave, parent, auto_update)
    160 

/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in __init__(self, gen, total, display, leave, parent, auto_update)
     56             self.leave,self.display=False,False
---> 57             parent.add_child(self)
     58         self.comment = ''

/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in add_child(self, child)
    208         self.inner_dict['pb2'] = self.child.progress
--> 209         self.show()
    210 

/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in show(self)
    215         self.html_code = '\n'.join([self.inner_dict[n] for n in to_show])
--> 216         self.out.update(HTML(self.html_code))
    217 

AttributeError: 'NBMasterBar' object has no attribute 'out'

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in fit(self, n_epoch, lr, wd, cbs, reset_opt)
    280                         self.epoch=epoch;          self('begin_epoch')
--> 281                         self._do_epoch_train()
    282                         self._do_epoch_validate()

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in _do_epoch_train(self)
    257         except CancelTrainException:                         self('after_cancel_train')
--> 258         finally:                                             self('after_train')
    259 

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in __call__(self, event_name)
    214 
--> 215     def __call__(self, event_name): L(event_name).map(self._call_one)
    216     def _call_one(self, event_name):

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in map(self, f, *args, **kwargs)
    362              else f.__getitem__)
--> 363         return self._new(map(g, self))
    364 

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in _new(self, items, *args, **kwargs)
    315     def _xtra(self): return None
--> 316     def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)
    317     def __getitem__(self, idx): return self._get(idx) if is_indexer(idx) else L(self._get(idx), use_list=None)

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __call__(cls, x, *args, **kwargs)
     41 
---> 42         res = super().__call__(*((x,) + args), **kwargs)
     43         res._newchk = 0

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __init__(self, items, use_list, match, *rest)
    306         if (use_list is not None) or not _is_array(items):
--> 307             items = list(items) if use_list else _listify(items)
    308         if match is not None:

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in _listify(o)
    242     if isinstance(o, str) or _is_array(o): return [o]
--> 243     if is_iter(o): return list(o)
    244     return [o]

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __call__(self, *args, **kwargs)
    208         fargs = [args[x.i] if isinstance(x, _Arg) else x for x in self.pargs] + args[self.maxi+1:]
--> 209         return self.fn(*fargs, **kwargs)
    210 

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in _call_one(self, event_name)
    217         assert hasattr(event, event_name)
--> 218         [cb(event_name) for cb in sort_by_run(self.cbs)]
    219 

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in <listcomp>(.0)
    217         assert hasattr(event, event_name)
--> 218         [cb(event_name) for cb in sort_by_run(self.cbs)]
    219 

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in __call__(self, event_name)
     20         "Call `self.{event_name}` if it's defined"
---> 21         if self.run: getattr(self, event_name, noop)()
     22 

/usr/local/lib/python3.6/dist-packages/fastai2/callback/progress.py in after_train(self)
     27     def begin_validate(self): self._launch_pbar()
---> 28     def after_train(self):    self.pbar.on_iter_end()
     29     def after_validate(self): self.pbar.on_iter_end()

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __getattr__(self, k)
    223             if attr is not None: return getattr(attr, k)
--> 224         raise AttributeError(k)
    225     def __dir__(self): return custom_dir(self, self._dir() if self._xtra is None else self._dir())

AttributeError: pbar

During handling of the above exception, another exception occurred:

AttributeError                            Traceback (most recent call last)
<ipython-input-8-495233eaf2b4> in <module>()
----> 1 learn.fit_one_cycle(4)

/usr/local/lib/python3.6/dist-packages/fastai2/callback/schedule.py in fit_one_cycle(self, n_epoch, lr_max, div, div_final, pct_start, wd, moms, cbs, reset_opt)
     96     scheds = {'lr': combined_cos(pct_start, lr_max/div, lr_max, lr_max/div_final),
     97               'mom': combined_cos(pct_start, *moms)}
---> 98     self.fit(n_epoch, cbs=ParamScheduler(scheds)+L(cbs), reset_opt=reset_opt, wd=wd)
     99 
    100 #Cell

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in fit(self, n_epoch, lr, wd, cbs, reset_opt)
    285 
    286             except CancelFitException:             self('after_cancel_fit')
--> 287             finally:                               self('after_fit')
    288 
    289     def validate(self, ds_idx=1, dl=None, cbs=None):

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in __call__(self, event_name)
    213     def ordered_cbs(self, cb_func:str): return [cb for cb in sort_by_run(self.cbs) if hasattr(cb, cb_func)]
    214 
--> 215     def __call__(self, event_name): L(event_name).map(self._call_one)
    216     def _call_one(self, event_name):
    217         assert hasattr(event, event_name)

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in map(self, f, *args, **kwargs)
    361              else f.format if isinstance(f,str)
    362              else f.__getitem__)
--> 363         return self._new(map(g, self))
    364 
    365     def filter(self, f, negate=False, **kwargs):

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in _new(self, items, *args, **kwargs)
    314     @property
    315     def _xtra(self): return None
--> 316     def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)
    317     def __getitem__(self, idx): return self._get(idx) if is_indexer(idx) else L(self._get(idx), use_list=None)
    318     def copy(self): return self._new(self.items.copy())

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __call__(cls, x, *args, **kwargs)
     40             return x
     41 
---> 42         res = super().__call__(*((x,) + args), **kwargs)
     43         res._newchk = 0
     44         return res

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __init__(self, items, use_list, match, *rest)
    305         if items is None: items = []
    306         if (use_list is not None) or not _is_array(items):
--> 307             items = list(items) if use_list else _listify(items)
    308         if match is not None:
    309             if is_coll(match): match = len(match)

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in _listify(o)
    241     if isinstance(o, list): return o
    242     if isinstance(o, str) or _is_array(o): return [o]
--> 243     if is_iter(o): return list(o)
    244     return [o]
    245 

/usr/local/lib/python3.6/dist-packages/fastai2/core/foundation.py in __call__(self, *args, **kwargs)
    207             if isinstance(v,_Arg): kwargs[k] = args.pop(v.i)
    208         fargs = [args[x.i] if isinstance(x, _Arg) else x for x in self.pargs] + args[self.maxi+1:]
--> 209         return self.fn(*fargs, **kwargs)
    210 
    211 #Cell

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in _call_one(self, event_name)
    216     def _call_one(self, event_name):
    217         assert hasattr(event, event_name)
--> 218         [cb(event_name) for cb in sort_by_run(self.cbs)]
    219 
    220     def _bn_bias_state(self, with_bias): return bn_bias_params(self.model, with_bias).map(self.opt.state)

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in <listcomp>(.0)
    216     def _call_one(self, event_name):
    217         assert hasattr(event, event_name)
--> 218         [cb(event_name) for cb in sort_by_run(self.cbs)]
    219 
    220     def _bn_bias_state(self, with_bias): return bn_bias_params(self.model, with_bias).map(self.opt.state)

/usr/local/lib/python3.6/dist-packages/fastai2/learner.py in __call__(self, event_name)
     19     def __call__(self, event_name):
     20         "Call `self.{event_name}` if it's defined"
---> 21         if self.run: getattr(self, event_name, noop)()
     22 
     23     @property

/usr/local/lib/python3.6/dist-packages/fastai2/callback/progress.py in after_fit(self)
     39     def after_fit(self):
     40         if getattr(self, 'mbar', False):
---> 41             self.mbar.on_iter_end()
     42             delattr(self, 'mbar')
     43         self.learn.logger = self.old_logger

/usr/local/lib/python3.6/dist-packages/fastprogress/fastprogress.py in on_iter_end(self)
    199             plt.close()
    200             self.out2.update(self.fig)
--> 201         total_time = format_time(time() - self.start_t)
    202         if self.text.endswith('<p>'): self.text = self.text[:-3]
    203         if self.total_time: self.text = f'Total time: {total_time} <p>' + self.text

AttributeError: 'NBMasterBar' object has no attribute 'start_t'

I already tried upgrading fastprogress but it was on the latest version. pip show has version 0.1.21

amritv · November 18, 2019, 7:18pm

Was going to post the same - I got the same and even loaded previous versions of fastprogress (0.1.20 and 0.1.19 just in case) but still same error

BTW I am using on Colab