If I had my way, downtown areas would have very low speed limits, and narrower streets would discourage cars from going fast. People who need cars (or, make deliveries) could still use them, but the streets would be slow enough that pedestrians and cyclist would feel seen and in-control.
Thankfully, this is exactly the sort of urban design we're gradually seeing more of, at least in many bigger European cities.
New UK Department for Transport guidance is for all urban and residential streets to have 20 mph (~30 km/h) speed limits, down from 30 mph previously. It is, however, still up to local authorities to actually set those limits. In London, most streets have been 20 mph for a while now.