Lesson 3 In-Class Discussion ✅


(Andrea de Luca) #487

Eight. Mhh.


(José Fernández Portal) #488

You can think of the convolutional layers as a “feature extractor” that detects a lot of patters in the images. Then the final linear layer, with one output per class, just matches certain patters from the “feature extractor” to specific classes. In this way you are reusing most of the network, instead of creating multiple single-classifications models. On top of that, I guess this approach captures relationship between classes (e.g., in the satellite images, if its “clear” it is not “cloudy”), that training independent models would be loosing. Finally, regarding the thresholds that you mention, I think you can set different values for each class.


#489

Is anyone else facing: https://forums.fast.ai/t/unet-binary-segmentation/29833/ . Any fix?


(Rick N) #490

Thanks; I had tried many different lr’s but hadn’t tried changing ps (dropout). Now I’ve done that, but even cutting ps down to 0.1 (which seems like a lot) doesn’t get train_loss < valid_loss. Here’s an example with resnet34, 24 epochs:

image

At this point I think I’ll just wait until we cover regularization in class so I know what I’m doing! I noticed in almost every example Jeremy showed last night train_loss was > valid_loss so it appears to be very common situation.


Lesson 2 In-Class Discussion ✅
#491

Between 1e-2 and 1e-1, you have 8 graduations for 2e-2, 3e-2, 4e-2, 5e-2, 6e-2, 7e-2, 8e-2 and 9e-2 :wink:


#492

Folders with . are hidden folders. you will not see them in Jupyter notebook browser. In terminal type ls -la


FAQ, resources, and official course updates ✅
(Ram Kripal) #493

Since every filter detect some feature, even if the image dimension increase they are able to detect the same feature.


(Elvin Wong) #494

probably because you’re running multiple jupyter nootbook in the same machine
run nvidia-smi
check in there are multiple python instances
±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1955 C /opt/anaconda3/bin/python 9075MiB |
| 0 3325 C /opt/anaconda3/bin/python 7191MiB |
±----------------------------------------------------------------------------+

If yes just shutdown the notebook and run it again


(Bilal) #495

Hi - I am stuck trying to change my classifier to a regression model. I am extracting labels from images in their filenames and want to use the labels as numbers and do a rmse as my loss function.

data = ImageDataBunch.from_name_func(path_img, fnames, label_func=get_float_labels, ds_tfms=get_transforms(),
                                     size=224, bs=bs)
data.normalize(imagenet_stats)

The labels come out as strings just fine ‘24’ etc. But if I try to change them to floats using a label_function, I get 0 classes.

My label function is a slight modification of the original:

np.random.seed(2)
pat = r'/([^/]+)_\d+.jpg

Also weird thing is I was at least able to get the strings converted to floats until today I did updated the fastai library. Not sure if something changed to cause this…

pat = re.compile(pat)
def get_float_labels(fn): return float(pat.search(str(fn)).group(1))


Also weird thing is I was at least able to get the strings converted to floats until today I did updated the fastai library. Not sure if something changed to cause this…

(Lankinen) #496

Test first convert into int and see uf that works.


(Bilal) #497

ints work! But floats do not - weird


(Bilal) #498

I guess I don’t care as long as it does the loss like I want it.

learn = create_cnn(data, models.resnet34)
learn.loss_func = MSELossFlat()

If I fit it though I get an error:
RuntimeError: Expected object of scalar type Float but got scalar type Long for argument #2 ‘target’


#499

dont you define it as MSELossFlat(input, target)?


(George Zhang) #500

Sorry for the imprecision here. I mean the point of steepest decline.


(Bilal) #501

I’m just following what was done in the lesson. There were no arguments given.


#502

I am getting this error when I try to run “Go Big” in camvid notebook, after training stage-1 and restarted kernel and ran datasets part. I am able to load stage-1 for stage-2 Go Big. I am able to run lr_find(learn) as well. When I run “learn.fit_one_cycle(10, slice(lr))” I get this errror. Any one got the same error or any help?

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-26-56c1c61458d6> in <module>()
----> 1 learn.fit_one_cycle(10, slice(lr))

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, wd, callbacks, **kwargs)
    20     callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor,
    21                                         pct_start=pct_start, **kwargs))
---> 22     learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
    23 
    24 def lr_find(learn:Learner, start_lr:Floats=1e-7, end_lr:Floats=10, num_it:int=100, stop_div:bool=True, **kwargs:Any):

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
    160         callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
    161         fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
--> 162             callbacks=self.callbacks+callbacks)
    163 
    164     def create_opt(self, lr:Floats, wd:Floats=0.)->None:

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
    92     except Exception as e:
    93         exception = e
---> 94         raise e
    95     finally: cb_handler.on_train_end(exception)
    96 

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
    87             if hasattr(data,'valid_dl') and data.valid_dl is not None:
    88                 val_loss = validate(model, data.valid_dl, loss_func=loss_func,
---> 89                                        cb_handler=cb_handler, pbar=pbar)
    90             else: val_loss=None
    91             if cb_handler.on_epoch_end(val_loss): break

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in validate(model, dl, loss_func, cb_handler, pbar, average, n_batch)
    52             if not is_listy(yb): yb = [yb]
    53             nums.append(yb[0].shape[0])
---> 54             if cb_handler and cb_handler.on_batch_end(val_losses[-1]): break
    55             if n_batch and (len(nums)>=n_batch): break
    56         nums = np.array(nums, dtype=np.float32)

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in on_batch_end(self, loss)
    236         "Handle end of processing one batch with `loss`."
    237         self.state_dict['last_loss'] = loss
--> 238         stop = np.any(self('batch_end', not self.state_dict['train']))
    239         if self.state_dict['train']:
    240             self.state_dict['iteration'] += 1

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in __call__(self, cb_name, call_mets, **kwargs)
    184     def __call__(self, cb_name, call_mets=True, **kwargs)->None:
    185         "Call through to all of the `CallbakHandler` functions."
--> 186         if call_mets: [getattr(met, f'on_{cb_name}')(**self.state_dict, **kwargs) for met in self.metrics]
    187         return [getattr(cb, f'on_{cb_name}')(**self.state_dict, **kwargs) for cb in self.callbacks]
    188 

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in <listcomp>(.0)
    184     def __call__(self, cb_name, call_mets=True, **kwargs)->None:
    185         "Call through to all of the `CallbakHandler` functions."
--> 186         if call_mets: [getattr(met, f'on_{cb_name}')(**self.state_dict, **kwargs) for met in self.metrics]
    187         return [getattr(cb, f'on_{cb_name}')(**self.state_dict, **kwargs) for cb in self.callbacks]
    188 

~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in on_batch_end(self, last_output, last_target, train, **kwargs)
    269         if not is_listy(last_target): last_target=[last_target]
    270         self.count += last_target[0].size(0)
--> 271         self.val += last_target[0].size(0) * self.func(last_output, *last_target).detach().cpu()
    272 
    273     def on_epoch_end(self, **kwargs):

TypeError: 'module' object is not callable

(Yash Mittal) #503

I am also getting same error


(Yash Mittal) #504

Change learn to:
learn = Learner.create_unet(data, models.resnet34, metrics=metrics).to_fp16()

and try to run. It will work


(Nick) #505

But filters are a fixed amount of pixels. So, for example, a filter detecting a black edge may just look like solid black when it is 4x bigger. Granted there are a range of filter sizes, and in photos things have to be learned at different zoom levels. But this won’t be the case for satellite images.


(Ram Kripal) #507

Since filter size not change only image size., When filter convolve with image, it detect particular wherever it exists in the image.