SFImage: A Swift Wrapper for FreeImage

(Stephen Johnson) #1

I believe Fast.ai is planning on using OpenCV for their image functionality for the Fast.ai in Swift library, but if anyone is interested, I’ve been working on a Swift wrapper for the FreeImage C library. You can find it here https://github.com/sjaz24/SFImage I’ve implemented copy on write semantics for it and so works like Swift arrays, etc.

Here’s a mixup type example. Load 2 images, rescale them to the same size, paste the one image onto the other, convert the image to float values between 0 and 1, then normalize the image with mean and standard deviation while extracting the data suitably for creating a tensor and lastly create the TensorFlow tensor from the data.

On my laptop, I can load a 64 image batch of dogs and a 64 batch of cats and do the below operation across multiple threads in about ~130 milliseconds.

if var image1 = SFImage(file: "/path/to/image/file"),
    var image2 = SFImage(file: "/path/to/other/image"),
    image1.rescale(to: (224, 224)),
    image2.rescale(to: (224, 224)),
    image1.paste(image: image2, alpha: 50),
    image1.convert(to: .rgbf) {

    let scalars = image1.getFlattenedAsFloat(with: (mean, std))
    let tensor = Tensor(shape: [3, image1.height, image1.width], scalars: scalars)
        // do something with the tensor
}

Similarly, cut up would be implemented something like:

if var image1 = SFImage(file: "/path/to/image/file"),
    var image2 = SFImage(file: "/path/to/other/image"),
    image2.cut(to: (50, 70, 100, 130)),
    image1.paste(image: image2, at: (65, 83), alpha: 255),
    image1.rescale(to: (224, 224)),
    image1.convert(to: .rgbf) {

    let scalars = image1.getFlattenedAsFloat(with: (mean, std))
    let tensor = Tensor(shape: [3, image1.height, image1.width], scalars: scalars)
        // do something with the tensor
}

The FreeImage installation on Mac is fairly painless but does need a couple of modifications. If anyone is interested in trying this out, let me know and I can provide more details.

Current Functionality

  1. Open image from file and save image to file
  2. Get image type, image height, width, bits per pixel and pitch
  3. Adjust gamma, brightness, contrast, colors and invert colors
  4. Paste one image or part of image onto another
  5. Cut - change image to the portion specified by rectangle
  6. Get pixel color and set pixel color
  7. Rotate image by specified degrees
  8. Combined operation of rotate by degrees around a specified origin and shift image with black mask or mirror background
  9. Flip horizontal and flip vertical
  10. Rescale/resize specifying filter method
  11. Rescale just a portion of the image and replace image with this rescaled portion
  12. Convert the image from one type to another. For example, RGB to Grayscale or from a RGB bitmap (pixel values 0 - 255) to a RGBF(loat) image (pixel values 0.0 - 1.0)
  13. Extract the image data as a 1 dimensional UInt8 array with the red/gray channel first, then green channel, then blue channel and optionally alpha channel. Can be used to convert the image to a TensorFlow tensor.
  14. Extract the image data as a 1 dimensional Float array with first red/gray channel, then green, then blue and optionally alpha channel. Optionally can apply mean and standard deviation normalization during extraction.
4 Likes