Changing substrate is a big rearrangement since you have already added the fish. You can actually get some new substrate and mix it up with the old gravel if you wish.
If you decide to switch the substrate completely, it can still be done if you do it carefully and quickly. You need to get everything prepared first. Including the new substrate. Yes, remove fish first is needed. You can temporary transfer them to a bucket or another fish tank. Let the filter run on whatever container you use to house them. Do not forget the heater and the air pump either. Especially the filter media, it must be wet the whole time even if you decide not to use it during the transfer... or you might risk losing the bacteria colonies.
Actually if I were to do it, it would be in this order.
1. Turn off all electronics first.
2. Drain the aquarium water to another fish tank or a water bucket (their temporary home). Leave 30% maybe a little less water in the main aquarium.
3. Transfer the fish to their temporary home.
4. Drain more water to almost just the substrate is left.
5. Remove the substrate.
6. Add the new substrate.
7. Add most of the water back first.
8. Fill the tank back up with new treated water.
9. Add the fish back (but do it the same way as you have just got them).