Eight. Mhh.
You can think of the convolutional layers as a “feature extractor” that detects a lot of patters in the images. Then the final linear layer, with one output per class, just matches certain patters from the “feature extractor” to specific classes. In this way you are reusing most of the network, instead of creating multiple single-classifications models. On top of that, I guess this approach captures relationship between classes (e.g., in the satellite images, if its “clear” it is not “cloudy”), that training independent models would be loosing. Finally, regarding the thresholds that you mention, I think you can set different values for each class.
1 Like
Thanks; I had tried many different lr’s but hadn’t tried changing ps (dropout). Now I’ve done that, but even cutting ps down to 0.1 (which seems like a lot) doesn’t get train_loss < valid_loss. Here’s an example with resnet34, 24 epochs:
At this point I think I’ll just wait until we cover regularization in class so I know what I’m doing! I noticed in almost every example Jeremy showed last night train_loss was > valid_loss so it appears to be very common situation.
1 Like
Between 1e-2 and 1e-1, you have 8 graduations for 2e-2, 3e-2, 4e-2, 5e-2, 6e-2, 7e-2, 8e-2 and 9e-2 
4 Likes
Folders with . are hidden folders. you will not see them in Jupyter notebook browser. In terminal type ls -la
Since every filter detect some feature, even if the image dimension increase they are able to detect the same feature.
probably because you’re running multiple jupyter nootbook in the same machine
run nvidia-smi
check in there are multiple python instances
±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 1955 C /opt/anaconda3/bin/python 9075MiB |
| 0 3325 C /opt/anaconda3/bin/python 7191MiB |
±----------------------------------------------------------------------------+
If yes just shutdown the notebook and run it again
Hi - I am stuck trying to change my classifier to a regression model. I am extracting labels from images in their filenames and want to use the labels as numbers and do a rmse as my loss function.
data = ImageDataBunch.from_name_func(path_img, fnames, label_func=get_float_labels, ds_tfms=get_transforms(),
size=224, bs=bs)
data.normalize(imagenet_stats)
The labels come out as strings just fine ‘24’ etc. But if I try to change them to floats using a label_function, I get 0 classes.
My label function is a slight modification of the original:
np.random.seed(2)
pat = r'/([^/]+)_\d+.jpg
Also weird thing is I was at least able to get the strings converted to floats until today I did updated the fastai library. Not sure if something changed to cause this…
pat = re.compile(pat)
def get_float_labels(fn): return float(pat.search(str(fn)).group(1))
Also weird thing is I was at least able to get the strings converted to floats until today I did updated the fastai library. Not sure if something changed to cause this…Test first convert into int and see uf that works.
1 Like
ints work! But floats do not - weird
I guess I don’t care as long as it does the loss like I want it.
learn = create_cnn(data, models.resnet34)
learn.loss_func = MSELossFlat()
If I fit it though I get an error:
RuntimeError: Expected object of scalar type Float but got scalar type Long for argument #2 ‘target’
1 Like
dont you define it as MSELossFlat(input, target)?
Sorry for the imprecision here. I mean the point of steepest decline.
1 Like
I’m just following what was done in the lesson. There were no arguments given.
I am getting this error when I try to run “Go Big” in camvid notebook, after training stage-1 and restarted kernel and ran datasets part. I am able to load stage-1 for stage-2 Go Big. I am able to run lr_find(learn) as well. When I run “learn.fit_one_cycle(10, slice(lr))” I get this errror. Any one got the same error or any help?
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-26-56c1c61458d6> in <module>()
----> 1 learn.fit_one_cycle(10, slice(lr))
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/train.py in fit_one_cycle(learn, cyc_len, max_lr, moms, div_factor, pct_start, wd, callbacks, **kwargs)
20 callbacks.append(OneCycleScheduler(learn, max_lr, moms=moms, div_factor=div_factor,
21 pct_start=pct_start, **kwargs))
---> 22 learn.fit(cyc_len, max_lr, wd=wd, callbacks=callbacks)
23
24 def lr_find(learn:Learner, start_lr:Floats=1e-7, end_lr:Floats=10, num_it:int=100, stop_div:bool=True, **kwargs:Any):
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in fit(self, epochs, lr, wd, callbacks)
160 callbacks = [cb(self) for cb in self.callback_fns] + listify(callbacks)
161 fit(epochs, self.model, self.loss_func, opt=self.opt, data=self.data, metrics=self.metrics,
--> 162 callbacks=self.callbacks+callbacks)
163
164 def create_opt(self, lr:Floats, wd:Floats=0.)->None:
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
92 except Exception as e:
93 exception = e
---> 94 raise e
95 finally: cb_handler.on_train_end(exception)
96
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in fit(epochs, model, loss_func, opt, data, callbacks, metrics)
87 if hasattr(data,'valid_dl') and data.valid_dl is not None:
88 val_loss = validate(model, data.valid_dl, loss_func=loss_func,
---> 89 cb_handler=cb_handler, pbar=pbar)
90 else: val_loss=None
91 if cb_handler.on_epoch_end(val_loss): break
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/basic_train.py in validate(model, dl, loss_func, cb_handler, pbar, average, n_batch)
52 if not is_listy(yb): yb = [yb]
53 nums.append(yb[0].shape[0])
---> 54 if cb_handler and cb_handler.on_batch_end(val_losses[-1]): break
55 if n_batch and (len(nums)>=n_batch): break
56 nums = np.array(nums, dtype=np.float32)
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in on_batch_end(self, loss)
236 "Handle end of processing one batch with `loss`."
237 self.state_dict['last_loss'] = loss
--> 238 stop = np.any(self('batch_end', not self.state_dict['train']))
239 if self.state_dict['train']:
240 self.state_dict['iteration'] += 1
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in __call__(self, cb_name, call_mets, **kwargs)
184 def __call__(self, cb_name, call_mets=True, **kwargs)->None:
185 "Call through to all of the `CallbakHandler` functions."
--> 186 if call_mets: [getattr(met, f'on_{cb_name}')(**self.state_dict, **kwargs) for met in self.metrics]
187 return [getattr(cb, f'on_{cb_name}')(**self.state_dict, **kwargs) for cb in self.callbacks]
188
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in <listcomp>(.0)
184 def __call__(self, cb_name, call_mets=True, **kwargs)->None:
185 "Call through to all of the `CallbakHandler` functions."
--> 186 if call_mets: [getattr(met, f'on_{cb_name}')(**self.state_dict, **kwargs) for met in self.metrics]
187 return [getattr(cb, f'on_{cb_name}')(**self.state_dict, **kwargs) for cb in self.callbacks]
188
~/anaconda3/envs/pytorch/lib/python3.7/site-packages/fastai/callback.py in on_batch_end(self, last_output, last_target, train, **kwargs)
269 if not is_listy(last_target): last_target=[last_target]
270 self.count += last_target[0].size(0)
--> 271 self.val += last_target[0].size(0) * self.func(last_output, *last_target).detach().cpu()
272
273 def on_epoch_end(self, **kwargs):
TypeError: 'module' object is not callable
1 Like
I am also getting same error
1 Like
Change learn to:
learn = Learner.create_unet(data, models.resnet34, metrics=metrics).to_fp16()
and try to run. It will work
1 Like
But filters are a fixed amount of pixels. So, for example, a filter detecting a black edge may just look like solid black when it is 4x bigger. Granted there are a range of filter sizes, and in photos things have to be learned at different zoom levels. But this won’t be the case for satellite images.
Since filter size not change only image size., When filter convolve with image, it detect particular wherever it exists in the image.