What exactly fish species did you put in your tank?
20 fish in a 40 gallon would be fine only if the tank is fully cycled before you adding the fish, and the fish must be tiny like no more than 2~3 inch long each.
You added too many fish at once...before the tank was cycled.
Filter must be turned on 24/7. Never turn it off.
Doing nitrogen cycle with fish in the tank is quite tricky. In order to keep the fish alive, you must do partial water change of 30~50% every day or at most every two days. But by doing so, you lower the ammonia concentration enough for the good bacteria to grow slower, yet not enough to be safe for your remaining 3 fish. What species are the last 3 fish anyway?
You must use a water conditioner on tap water before you add it to the fish tank. Or fish will die to chlorine in the tap water.
I recommend Prime, as it is not only a water conditioner that neutralize chlorine like the typical aquarium water conditioners do, but also it can detoxify ammonia for up to 48 hours.
Never change 100% the water. Fish dislike sudden large change in water temperature, hardness, PH, nitrate concentration etc. Do no more than 50% partial water change will lesser the amount of changes.
Are you referring to C or F when you said 35 to 40 degree? If it is C, then the temperature is too high.