Yes, you can do this by treating it as a regression problem. Here is an example that goes into a lot of detail so I’ll highlight the main part that shows this type of regression—notice that in DataBlock the blocks parameter is given an ImageBlock for the inputs and RegressionBlock for the outputs, in this case age is predicted by the image:
2 Likes
Awesome! That was exactly what I was referring to! Thanks for sharing this @vbakshi !
To dive into this a little bit deeper: Does somebody know if there exists a procedure for multidimensional targets that are statistically dependent to each other?
So assume instead of just 1 knob for temperature like in the example of my original post 2 days ago, we now would have to analyze images of stormy clouds. Our target consists of 3 knobs; 1 knob for guessing the temperature, 1 knob for guessing the pressure, 1 knob for guessing humidity. All of these 3 variables passively depend on each other, e.g. with higher temperature, there might be higher humidity as well. So how would you calculate a loss in this case? It’s like a multidimensional loss with more than 1 target, where the targets are not independent of each other - is something like this possible?
1 Like
I don’t know how the dependency of the different outputs on each other would affect the loss calculation, but this Walk with fastai example might answer your question on how to train multiple targets for each image.
1 Like
Is there a GitHub repository for the course videos? I’m particularly interested in the code for Lesson 13.
Cycle-gan from 2018? I think it is in this repo. https://github.com/fastai/course-v3/tree/master/nbs/dl2
I just assumed that maybe in case of dependency the loss of the multiple targets could also be calculated by using co variance matrices, but thanks again @vbakshi the walk with fastai example with multiple targets in an image was also exactly what I had in mind ! Thanks for sharing this !
Another question: does something like this exist / would it be possible to design a model that learns how to set some of these knobs? Something like a parameter / tensor / knob setting assistance tool?
So say you’d have a software for image or video editing or an digital audio workstation or any software that has knobs to set certain parameters to edit something. You’d create a lot of data for nearly unprocessed video / audio files / whatever and have a model X to learn the multiple targets (multiple knobs inside the software that are all pretty low in the first model X, so the dry/wet ratio would be something like 95/5, very little effect of the processing visible in the image/audio/etc).
Then you’d train a model Y with the exact same files / data like in model X, but with the difference that all of the data has been modified so that the processing of the software is set just about right. So in model Y for the exact same targets you would get a dry/wet ratio of maybe 50/50. Meaning that the targets in model X are all lower in value than the targets of model Y but essentially they are describing the same physical parameter. So in the final step you would sort of “measure the distance between the input data points of model X and model Y and learn what effect it has on the targets of model X and model Y” and put that information inside yet another model to have it learn how to get from data points X to data points Y buy changing targets X to targets Y. It’s like a model that watches inputs and outputs of two other models and learns how to get from one to another. Could this be done with two models that have multiple targets? Then you could use that third final model for uncompressed/unprocessed data to have kind of an “assistance parameter setting tool” where it would suggest certain settings inside the software automatically? Would something like this be possible?
There should be some ‘import’ lines before this. If working through Kaggle notebook, ensure you have run code in the previous cells.
Kaggle sometimes collapses code cells for brevity. e.g
‘’’
from duckduckgo_search import ddg_images
from fastcore.all import *
def search_images(term, max_images=30):
print(f"Searching for ‘{term}’")
return L(ddg_images(term, max_results=max_images)).itemgot(‘image’)
‘’’
it worked! , there was some other issue
thanks!