Align two images

dipam7 · August 26, 2019, 10:03pm

I have a problem in which I have two images rendered by a model. These are satellite images. One image is a wide view of an area and the other image is a small part of it. I want to crop the wide image in a way that it aligns with the small part. Any ideas on how to solve this problem?

brunosan · August 26, 2019, 10:52pm

I´m assuming you don´t know exactly where the crop part goes, and they don´t align perfectly.

I would slide the small crop over the big part until I find the biggest cross-correlation (it´s like a sum of the differences). Once if I find the approximate spot, I´d do a crop with a rim, and then find a affine transform to minimze the difference (shift, plus perspective, plus warp).

For the first part, this seems to be place to look: Template Matching with OpenCV
For the second, this: https://scikit-image.org/docs/dev/auto_examples/transform/plot_matching.html

If you are looking for Deep Learning ways to do this…

I wonder if the first part could use a case similar to the headpose lesson: I don’t know how to add two input images, but once you do that, you can create infinite training data cropping a piece of an image and then adding some transform to the crop. The regression would have the big and cropped images as input, and the location of the center of the crop as output.
For the second part, once you have to different versions of the small cropped region, may be with a GAN or similar that modifies the small image so the classifier cannot make up the difference?

Hope it helps,

Bruno

dipam7 · August 27, 2019, 2:13pm

Hey, thank you for the response. I will look into template matching and scikit-image links. I’m flexible with the implementation, doing it using deep learning is not a necessity. Also, the images don’t have to be transformed, that makes my life easier! Thanks again.

sayakgis · August 27, 2019, 4:18pm

Hi Dipam,

Cosmiq has a library called solaris, which handles satellite images in excellent and efficient way. It’s a high level library so less coding.

https://solaris.readthedocs.io/en/latest/

Another way is to crop bigger image to small tiles using numpy array functions and save them with row & col index, which can be used later to mosaic later to bigger image.

github.com

marcsv87/Solar-Panels-Detection/blob/master/src/data_loader.py

import torch.utils.data as data

from PIL import Image
import os
import os.path
#from tifffile import imread
from scipy.misc import imread, imresize
import torchvision.transforms as transforms
import torch
import numpy as np
import cv2
import random
import tifffile as tiff

IMG_EXTENSIONS = [
    '.jpg', '.JPG', '.jpeg', '.JPEG',
    '.png', '.PNG', '.ppm', '.PPM', '.bmp', '.BMP',
]

def is_image_file(filename):

This file has been truncated. show original

Hope this helps.
Thanks-Sayak

digitalspecialists · August 27, 2019, 5:58pm

You haven’t really explained your problem completely, but if it is finding where an image was cropped, especially if tiled, then solutions include: template matching, comparing hashes, dice/jaccard scores, edge matching.

I use the last of these a lot. If you know one image is inside or adjacent to another and close to pixel equivalence, you can flatten the channels on an edge as the 1D ‘needle’ and compare it to same size strips of the ‘haystack’, comparing the distance between each element of the 1D arrays.