In chapter 13 of the fastbook, Jeremy writes:
A stride 2 conv with the default padding (1) and ks (3) will reduce the activation map dimension by half. Formula: (n + 2*pad - ks)//stride + 1. As the activation map dimension reduces by half we double the number of filters. This results in no overall change in computation as the network gets deeper and deeper.
Why is it that we have to double the number of filters when the activation map dimension reduces by half?
You might saw it already 
The reason is explained later in the chapter:
We can now use this information to clarify our statement in the previous section: “When we use a stride-2 convolution, we often increase the number of features because we’re decreasing the number of activations in the activation map by a factor of 4; we don’t want to decrease the capacity of a layer by too much at a time.”
There is one bias for each channel. (Sometimes channels are called features or filters when they are not input channels.) The output shape is
64x4x14x14, and this will therefore become the input shape to the next layer. The next layer, according tosummary, has 296 parameters. Let’s ignore the batch axis to keep things simple. So for each of14*14=196locations we are multiplying296-8=288weights (ignoring the bias for simplicity), so that’s196*288=56_448multiplications at this layer. The next layer will have7*7*(1168-16)=56_448multiplications.What happened here is that our stride-2 convolution halved the grid size from
14x14to7x7, and we doubled the number of filters from 8 to 16, resulting in no overall change in the amount of computation. If we left the number of channels the same in each stride-2 layer, the amount of computation being done in the net would get less and less as it gets deeper. But we know that the deeper layers have to compute semantically rich features (such as eyes or fur), so we wouldn’t expect that doing less computation would make sense.