@wgpubs I think you’re misunderstanding. The code you quoted is used only to train the model, not for inference.
Also, the docs you’re quoting from have not yet been updated for the new single image API.
Ok I think I got you now.
So …
data2 = ImageDataBunch.single_from_classes(path, data.classes, tfms=tfms, size=224).normalize(imagenet_stats)
learn = create_cnn(data2, models.resnet34)
learn.load('one-epoch')
… could be re-written as …
data2 = ImageDataBunch.single_from_classes(path, ['dog','cat'], tfms=tfms, size=224).normalize(imagenet_stats)
learn = create_cnn(data2, models.resnet34)
learn.load('one-epoch')
I think I was getting hung up on your use of the data variable.
Also, what is this line for … data2.valid_ds.set_item(img)?
Also, also, is this going to play nicely with multilabel datasets? Or is the idea to tackle that when it comes up later?
1 Like
That’s me experimenting - I’m thinking that’s what I’ll use internally soon…
I got you.
I’m still partial to something more akin to what I’ve proposed … ImageDataBunch.single_from_classes to get a DataBunch seems strange to me API wise.
Nevertheless, I imagine either approach can be generalized for other applications (text, tabular, etc…) if we change things up a bit. This should work with either what you are suggesting or what I have.
For example, something like this for image classification …
learn = create_cnn(data=None, models.resnet34, pretrained=false, tfms=tfms, size=224, n_classes=2)
learn.load('stage2-34.pth')
img = open_image(PATH/'valid/in/112.jpg')
input = learn.prepare_example(img)
learn.predict(input)
… and something like this for tabular data based classification …
learn = get_tabular_learner(data=None, layers=[200,100], tfms=tfms, size=224, n_classes=2)
learn.load('stage-2.pth')
df = pd.read_csv(path/'adult.csv')
input = learn.prepare_example(df)
learn.predict(input)
prepare_example would be an abstract methond on Learner that each Learner subclass could implement to apply whatever transforms, etc… are needed to a single example in order to run it through it’s model.
Currently we don’t really have a way to serialize all the pipeline for a learner. So you need to create it. Your approach would be a nice API, but it could be rather tricky to implement in a way that supports all the possible pipelines and applications!
Let’s move followups to this thread since this isn’t specific to the course:
Hi @jeremy! Since so many people want to do this, I’ve created a simple library that serves a FastAI Learner object and gives a nice web-based UI with a single line of code: https://github.com/aakashns/servefastai
Here’s a demo:
Currently it’s more a proof-of-concept for playing around with the model and passing custom test images, and only works for single-label image classification but I’m hoping to make it production ready by the end of the course.
Hope this is useful! 
16 Likes
Pretty slick!
1 Like
Damn. Pretty awesome!
1 Like
Would be great to implement something similar as a part of a standard pipeline, like, inside of Jupyter notebook. Though it is probably more difficult. I am thinking about something similar to bokeh that uses JS to integrate various fancy plots into a notebook.
@aakashns Probably we can come up with a notebook extension, or just use HTML() function from Jupyter to render right into the notebook? Or using magic like %show_learner?
Not sure which approach is better. Maybe some CLI stuff also? Though probably fastai team already has something in mind.
1 Like
Also, I think your solution is a bit different than the original question.
AFAIU, the original question was about how to load a trained fastai model from disk with minimum dependencies to serve the model for inference in production pipelines.
But your thing is serving a model/learner already in available in memory I think. Needless to say what you have done is super useful too!
Also, I started looking at your PyTorch screencasts. They are super awesome. Very clear explanations to understand the PyTorch API. Can wait to binge the rest of the videos
1 Like
Yo @simonw … do you know of any way to configure things so zeit spins up the machine faster? I’ve deployed a dockerized flask app and the requests usually timeout while the server spins up. I’ve also seen that you, and a few others, are using starlette.io, and I was wondering how that compares with flask (especially w/r/t performance).
Anyways, any tips of making my API spin up quicker and choosing a framework to serve the API would be greatly appreciated.
-wg
Zeit’s v2 platform spins up a lot faster… but it comes at a nasty cost: your image must be less than 100MB. If you’re bundling a 85MB model that’s likely impossible. There’s a discussion about that issue here: https://github.com/zeit/now-cli/issues/1523
The easier option is to tell Zeit to always run that particular instance - then you won’t have to pay the startup cost. You can do that using the scale command: https://zeit.co/blog/scale - the free plan lets you have up to 3 instances running at a time, and you can pay them money for more.
1 Like
Thanks for the solid tips!
What about starlette via flask? Any opinion on which is better and/or when to use one over the other?
Yeah, I’m pretty sure it can be embedded into Jupyter itself using the HTML() function. Haven’t tried it yet though. Don’t know much about Jupyter extensions though.
Just learned about starlette. Seems quite nice, especially the fact that it’s asychronous!
Lecture 2 shows a great example of how to build a widget for IPython notebook. I guess your app could be converted into widget then. There are a lot of pre-built HTML controls in the ipywidgets package.
1 Like
Great idea! I had the exact same thought after watching the FileDeleter example. I’ll try and submit a pull request to the fastai library’s widgets section.
1 Like
When I tried to run learn.model(img.data) . The model expect number of input is equal to the batch size then I got the error below
RuntimeError: Expected 4-dimensional input for 4-dimensional weight [64, 3, 7, 7], but got 3-dimensional input of size [3, 28, 28] instead
Could you help me to figure out how to run learn.model with a specific image or a bunch of image ?
Thank you
Maybe you can feed in the image when you add 1 dim as dummy bs=1 to make it 4D, i.e. [1, 3, 28, 28]?
1 Like