This competition looks very interesting and I think the solution can have immense humanitarian uses (disaster response, urban planning, census etc). Although, I am not sure how can we leverage powerful image feature detection capabilities of CNNS but also tackle the challenges of mutli class detection and the continuity of the images which is very different from image classifications we did in the course (cats & dogs or fisheries). Seems like FCNs are the way to go?
Regarding FCNs, I found this article very useful: https://devblogs.nvidia.com/parallelforall/image-segmentation-using-digits-5/
Any other must know ideas/articles about working with satellite imagery and object detection?
I’m working on this with a friend, and my impressions so far that there’s quite a bit of “engineering” of the image data.
so far:
1)all the .tif files are layered images with different bands of information and the 16-band and 3 band .tif images need to be unpeeled as a separate image layer, we’re using the .png format
2) you have to find a way to read in the csv data for the training data that maps the cooordinates onto the training images so the net knows what a “tree” looks like from a satellite view as polygons
3) each “region” is actually composed of 5 different adjacent files so the five files need to be aligned together so the analysis doesn’t miss what’s at the seams of the images.
4) the 3-band and 16-band images are at different resolutions and should be normalized against each other.
5) the different bands of data need to be “registered” or aligned on top each other as the different bands may be shifted
Just querying the void: is there any value in using a pretrained model from vgg() for this image challenge? Vgg() knows about how to process color images, does that mean it only knows about RGB bands? The satellite images in total have about 20 bands of information. And a car from the birds’ eye angle doesn’t look like a car from an image-net image. Or would it simply take too long for a model to learn from scratch?
From a data pre-processing perspective, playing with these data satellite images is very storage intensive. The original tiff files are uint16, and after some of the preprocessing the numpy arrays are coming out as 200gig files, (I’ve had to enlarge the EC2 instances volume twice from the standard storage allotment). We starting to save image data as HDF5 files to help manage file size b/c this file format compresses data as it stores it.
The friendly warning for people trying out this challenge is that the HDF5 format is fragile, especially when it’s in the middle of writing the file. If you accidentally stop the write/read process midway (say closing the wrong terminal window accidentally) the entire file can be corrupted and you have to rebuild the data from the images again.
I’ve been working on a similar problem, and a cool approach is outlined in this paper “Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping”: https://arxiv.org/pdf/1510.00098.pdf
tldr: researches trained vgg 8 on ImageNet; then finetuned the model to predict nighttime light intensity from daytime images, which has the correlated result of optimizing for filters that pick out roads, urban areas, farms, etc; then this is used to predict poverty levels with ~70% accuracy compared to ground-level surveys
One interesting issue that didn’t come up in part 1 of the class but is significant for this challenge is reweighting the loss function. In this competition is that the model gets the most “reward” for classifying large areas like crops and roads, and then the model has little incentive to identify smaller objects that don’t occupy a large area like trucks and walls. A strategy that some competitors are trying is training different models on different classes separately.
[this post has been edited from an earlier post that had outlined a problem that ended up as a data processing error, and not a model error. However, I thought remaining part about imbalanced datasets and reweighting might still be useful to the curious. ]
Hi Everyone,
I am quite fascinate by this idea of detecting object through satellite images. I am new to image processing and deep learning, and I dont have much idea about how to deal with satellite images and how to use deep learning to identify features.
There are lot of terms in the competition that I dont understand like 16 band images, polygons etc.
Can someone point me to some resource about satellite imagery and how to process them, just like a lesson 0 (starting point).
The DSTL Kaggle competition just closed. And our 2-person team placed 22nd! The top 6% of the current competitors.
I’m hoping to take the notes I posted here eventually and our data pipeline code and bundle them together for some kind of startup guide for beginners to start playing with satellite data.
Neither of us knew how to deal with satellite data before the competition, entered a Kaggle competition, or wrote a model that does image segmentation/ semantic segmentation/ image segmentation + classification (I don’t know the right buzzwords!). But because of this class and the Kaggle’s public community, we (quickly!) learned enough to be competitive! Really big thanks @jeremy and @rachel for making democratizing the tech and nurturing this community !!!
It was very much a surprise b/c we were 123rd on the public leaderboard when the competition closed on the public dataset (19% of the total data), and then found out after the rest of the 81% of the data was calculated our prediction accuracy on the different classes of objects went up, while the leaders’ accuracy went down. We’re still trying to figure why/how that happened or if that is normal for Kaggle competitions.
I can’t wait to hear about your experiences. There’s very little material out there to help people do well in Kaggle competitions with very little previous ML background, so I think your story and tips will be helpful for many people!
