FastAPI / Starlette issue: AttributeError: '_FakeLoader' object has no attribute 'noops'

So attempting to put the bears classifier behind a FastAPI/Starlette backend but have been struggling with this error for the past few hours. Any ideas???"/predict")
async def predict(image: UploadFile=File(...)):
    img_bytes = await
    pred_class, pred_class_idx, probs = inf_learn.predict(img_bytes)

starts recursively throwing this error:

api-dev_1   |   Traceback (most recent call last):
api-dev_1   |   File "<string>", line 1, in <module>
api-dev_1   |   File "/usr/local/lib/python3.7/multiprocessing/", line 105, in spawn_main
api-dev_1   |     exitcode = _main(fd)
api-dev_1   |   File "/usr/local/lib/python3.7/multiprocessing/", line 115, in _main
api-dev_1   |     self = reduction.pickle.load(from_parent)
api-dev_1   | AttributeError: '_FakeLoader' object has no attribute 'noops'

I’ve tried several ways to create the input for inf_learn.predict including converting img_bytes to a numpy array and a PILImage … all of them throw the same exception.

1 Like

So I pieced this out into a much simpler application … and while it’s still not working, I was able to get a better stack trace.

Any ideas?

Here again is the call:"/predict")
async def predict(image: UploadFile=File(...)):
    img_bytes = await
    pred_class, pred_class_idx, probs = inf_learn.predict(img_bytes)

    return {
        "predicted_class": pred_class,
        "probability": classes[probs]

… and after all the complaints about _FakeLoader not having a noops attribute, I get this:

INFO: - "POST /predict HTTP/1.1" 500 Internal Server Error                                       
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/uvicorn/protocols/http/", line 385, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/uvicorn/middleware/", line 45, in __call__
    return await, receive, send)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastapi/", line 149, in __call__
    await super().__call__(scope, receive, send)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/", line 102, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/middleware/", line 181, in __call__
    raise exc from None
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/middleware/", line 159, in __call__
    await, receive, _send)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/", line 82, in __call__
    raise exc from None
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/", line 71, in __call__
    await, receive, sender)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/", line 550, in __call__
    await route.handle(scope, receive, send)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/", line 227, in handle
    await, receive, send)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/starlette/", line 41, in app
    response = await func(request)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastapi/", line 166, in app
    dependant=dependant, values=values, is_coroutine=is_coroutine
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastapi/", line 119, in run_endpoint_function
    return await**values)
  File "./", line 40, in predict
    pred_class, pred_class_idx, probs = inf_learn.predict(img_bytes)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/", line 231, in predict
    inp,preds,_,dec_preds = self.get_preds(dl=dl, with_input=True, with_decoded=True)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/", line 219, in get_preds
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/", line 178, in _do_epoch_validate
    dl,*_ = change_attrs(dl, names, old, has);       self('after_validate')
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/", line 124, in __call__
    def __call__(self, event_name): L(event_name).map(self._call_one)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastcore/", line 372, in map
    return self._new(map(g, self))
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastcore/", line 323, in _new
    def _new(self, items, *args, **kwargs): return type(self)(items, *args, use_list=None, **kwargs)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastcore/", line 41, in __call__
    res = super().__call__(*((x,) + args), **kwargs)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastcore/", line 314, in __init__
    items = list(items) if use_list else _listify(items)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastcore/", line 250, in _listify
    if is_iter(o): return list(o)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastcore/", line 216, in __call__
    return self.fn(*fargs, **kwargs)
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/", line 127, in _call_one
    [cb(event_name) for cb in sort_by_run(]
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/", line 127, in <listcomp>
    [cb(event_name) for cb in sort_by_run(]
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/callback/", line 24, in __call__
    if and _run: getattr(self, event_name, noop)()
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/callback/", line 95, in after_validate
    if self.with_input:     self.inputs  = detuplify(to_concat(self.inputs, dim=self.concat_dim))
  File "/Users/admin/Development/_tools/anaconda/envs/fastapi_test_app/lib/python3.7/site-packages/fastai2/", line 213, in to_concat
    if is_listy(xs[0]): return type(xs[0])([to_concat([x[i] for x in xs], dim=dim) for i in range_of(xs[0])])
IndexError: list index out of range
1 Like

Hey I’m doing something similar* and have it working. Maybe it can help:

*: loads a model pkl and serves predictions using Flask. posts images to the server from a running webcam using OpenCV.
I need to run fastai2 in my linux vm (WSL) but need to run the client in windows to have hardware support for the webcam.

Nice setup there!

Unfortunately, I’m not getting everything back from learn.predict though the code is almost identical (I’m using FastAPI/Starlette).

from fastapi import FastAPI, File, UploadFile
from import *

path = Path(__file__).parent
app = FastAPI()

inf_learn = load_learner(path/'export.pkl')
classes = list(inf_learn.dls.vocab)"/predict")
async def predict(image: UploadFile=File(...)):
    img_bytes = await
    pred_class, pred_idx, probs = inf_learn.predict(img_bytes)

    return {
        "class": pred_class,
        "pred_idx": pred_idx,
        "probs": probs,
        "classes": classes

Here’s the weird part.

It returns the pred_class just fine … but for everything else, it returns an empty dictionary:

    "class": "grizzly",
    "pred_idx": {},
    "probs": {},
    "classes": [

No errors … but no pred_idx or probs.

Hmm, I’d suggest trying to isolate the problem between the BytesIO aspect and fastai, by replicating my ugly process of bringing in numpy and cv2 to decode the request data. If that works, you know the problem is in encode/decode step, but if that doesn’t work, it points towards fastai’s learner.predict method being off.

If there was a problem with the input data (bytes in my case though I’ve tried others), why would I be getting the predicted class returned correctly?

It seems to point to something wrong with the learner.predict method

Can you try it in a notebook regularly?

Works just fine (just using the ch.2 code out of the box).

I’ve also tried passing a file path to my inference learner in my FastAPI app … I get the same results (like we do in the chapter 2 notebook)


How this was crazy and weird.

So the problem was not with learner.predict but with how FastAPI (and maybe Starlette) behaves when attempting to return a type that is a pytorch tensor. For whatever reason, it doesn’t throw an error … it just says, “Oh I see you have something I’m not sure about … I’ll give you an empty dictionary”.

So the fix is this:

    return {
        "class": pred_class,
        "pred_idx": pred_idx.item(),
        "probs": probs.tolist(),
        "classes": classes

pred_ix and probs are both tensor objects … and FastAPI doesn’t like them. Unfortunately, instead of throwing an exception, it just converts them to {}.

With the fix in place above all works. Here’s an example of what I get:

    "class": "teddy",
    "pred_idx": 2,
    "probs": [
    "classes": [