What is a substitute for DatasetFormatter() in Fastai v2

Part 1 (2019) Advanced (Part 1 v3)
1

Hello, I’ve been using DatasetFormatter() class from vision widgets for the purpose if determining image duplicates in a dataset, link to the class: https://fastai1.fast.ai/widgets.image_cleaner.html
However, this functionality seems to be removed in Fastai v2? I am currently using fastai 2.2.7. Is there a different class in place of DatasetFormatter()? Or what functionality will allow me to check dataset for duplicate images. Would appreciate if you point me to appropriate resources.

Thanks,

2

The most closest that I can find is https://docs.fast.ai/vision.widgets.html#ImageClassifierCleaner but I cannot find an equivalent of find_similars. This may require implementing custom functionality provided by find_similars https://github.com/fastai/fastai1/blob/master/fastai/widgets/image_cleaner.py#L40.

- get activations from the specified layers
- get similarity between the two
- sort indices based on similarities