I believe Fast.ai is planning on using OpenCV for their image functionality for the Fast.ai in Swift library, but if anyone is interested, I’ve been working on a Swift wrapper for the FreeImage C library. You can find it here https://github.com/sjaz24/SFImage I’ve implemented copy on write semantics for it and so works like Swift arrays, etc.
Here’s a mixup type example. Load 2 images, rescale them to the same size, paste the one image onto the other, convert the image to float values between 0 and 1, then normalize the image with mean and standard deviation while extracting the data suitably for creating a tensor and lastly create the TensorFlow tensor from the data.
On my laptop, I can load a 64 image batch of dogs and a 64 batch of cats and do the below operation across multiple threads in about ~130 milliseconds.
if var image1 = SFImage(file: "/path/to/image/file"),
var image2 = SFImage(file: "/path/to/other/image"),
image1.rescale(to: (224, 224)),
image2.rescale(to: (224, 224)),
image1.paste(image: image2, alpha: 50),
image1.convert(to: .rgbf) {
let scalars = image1.getFlattenedAsFloat(with: (mean, std))
let tensor = Tensor(shape: [3, image1.height, image1.width], scalars: scalars)
// do something with the tensor
}
Similarly, cut up would be implemented something like:
if var image1 = SFImage(file: "/path/to/image/file"),
var image2 = SFImage(file: "/path/to/other/image"),
image2.cut(to: (50, 70, 100, 130)),
image1.paste(image: image2, at: (65, 83), alpha: 255),
image1.rescale(to: (224, 224)),
image1.convert(to: .rgbf) {
let scalars = image1.getFlattenedAsFloat(with: (mean, std))
let tensor = Tensor(shape: [3, image1.height, image1.width], scalars: scalars)
// do something with the tensor
}
The FreeImage installation on Mac is fairly painless but does need a couple of modifications. If anyone is interested in trying this out, let me know and I can provide more details.
Current Functionality
- Open image from file and save image to file
- Get image type, image height, width, bits per pixel and pitch
- Adjust gamma, brightness, contrast, colors and invert colors
- Paste one image or part of image onto another
- Cut - change image to the portion specified by rectangle
- Get pixel color and set pixel color
- Rotate image by specified degrees
- Combined operation of rotate by degrees around a specified origin and shift image with black mask or mirror background
- Flip horizontal and flip vertical
- Rescale/resize specifying filter method
- Rescale just a portion of the image and replace image with this rescaled portion
- Convert the image from one type to another. For example, RGB to Grayscale or from a RGB bitmap (pixel values 0 - 255) to a RGBF(loat) image (pixel values 0.0 - 1.0)
- Extract the image data as a 1 dimensional UInt8 array with the red/gray channel first, then green channel, then blue channel and optionally alpha channel. Can be used to convert the image to a TensorFlow tensor.
- Extract the image data as a 1 dimensional Float array with first red/gray channel, then green, then blue and optionally alpha channel. Optionally can apply mean and standard deviation normalization during extraction.