Cohen's Kappa score

Hello all,

I would like to use the Cohen’s Kappa score as a metric (playing with the Diabetic Retinopathy dataset)

I tried the following:

learn = create_cnn(data, models.resnet34, metrics=[error_rate, KappaScore()])

but at the end of the epoch I got:

RuntimeError: Expected object of backend CUDA but got backend CPU for argument #2 'other'

How can I get the Cohen’s Kappa score to work?

Try the following :

learn = create_cnn(data, models.resnet34, metrics=[error_rate, KappaScore])

You want to pass reference to the function, or in this case the class.

I get this:

TypeError: __init__() takes from 1 to 2 positional arguments but 3 were given

Where is that error coming from ? The KappaScore function ?

This is when I run

Note that KappaScore is a class, not a function…

Here’s a link for it:

This error means something involved in your function is on the CPU when it should be on the GPU.

How can this be fixed? This is a fastai function so is there a bug with the fastai function?

This was fixed by upgrading the version of fastai in Kaggle Kernels (the platform that I am using)… turns out that fastai 1.0.39 has this bug, which I guess has to do with the confusion matrix because this problem happened with FBeta as well which depends on the ConfusionMatrix class like KappaScore… but fastai 1.0.40 has it fixed…


The code I use (and which is working well) is (fastai 1.0.50.post1) :

kappa = KappaScore()
kappa.weights = "quadratic"
learn = cnn_learner(data, models.resnet34, metrics=[error_rate, kappa])

hi all… I have fastai ‘1.0.53.post2’ version but still i have the same problem with KappaScore as metrics
TypeError: __init__() takes from 1 to 2 positional arguments but 3 were given

so i cannot figure out what is going wrong.

Some code below:
kappa = KappaScore()
kappa.weights = “quadratic”
learn = cnn_learner(data, models.resnet50, metrics=kappa)

I got a RuntimeError:Expected object of scalar type Float but got scalar type Long for argument #2 ‘other’
How can i solved it?

1 Like

I have the same error. any advice please?

I have the same error message but it is due to an error creating the ImageList