sharing this in case it is helpful for others. Would love to add content like this to the lesson notes wiki…
1. use kg config/kg download to get train.zip and test.zip. unzip them both, and delete the .zips
2. move 2000 random images from ./train to ./valid
3. move 200 random images from ./train to ./sample/train
4. move 50 random images from ./valid to ./sample/valid
5. create 'cats' and 'dogs' subdirs in ./train, ./valid, ./sample/train, and ./sample/valid dirs
6. move all cat/dog files in ./train, ./valid, ./sample/train, and ./sample/valid to the respective 'cats/dogs' subdirs created in the previous step
7. create ./test/unknown subdir
8. move all image files in ./test to ./test/unknown
#dir structure summary
.
├── results
├── test
│ └── unknown
├── train
│ ├── cats
│ └── dogs
├── valid
│ ├── cats
│ └── dogs
└── sample
├── results
├── test
├── train
│ ├── cats
│ └── dogs
└── valid
├── cats
└── dogs
#dir structure detail
.
├── results #saved models/weights for the main set
├── sample #a smaller replica of the main set, to use for quick testing while developing
│ ├── results #saved models/weights for the sample set
│ ├── test
│ ├── train # 200 random, moved from ../train
│ │ ├── cats # ~100 cat images (cat.1234.jpg), moved from its parent directory
│ │ └── dogs # ~100 dog images (dog.1234.jpg), moved from its parent directory
│ └── valid # 50 random, moved from ../valid
│ ├── cats # ~25 cat images (cat.1234.jpg), moved from its parent directory
│ └── dogs # ~25 dog images (dog.1234.jpg), moved from its parent directory
├── test # original test.zip from kaggle, unzipped. 12500 unclassified (e.g. 1234.jpg)
│ └── unknown
├── train # orginal train.zip from kaggle, unzipped. 25000 classified (e.g. dog.1234.jpg)
│ ├── cats # ~11400 cat images (cat.1234.jpg), moved from its parent directory
│ └── dogs # ~11400 dog images (dog.1234.jpg), moved from its parent directory
└── valid # 2000 random, moved from ./train
├── cats # ~975 cat images (cat.1234.jpg), moved from its parent directory
└── dogs # ~975 cat images (cat.1234.jpg), moved from its parent directory