The most feelbad part of Kai'Sa's kit is her Icathian Rain. It's guaranteed damage that had ZERO counter-play. All she has to do is get near you, press Q and deliver a ton of damage. It acts exactly like Sona's Q but with one HUGE difference that should introduce counterplay. The difference is that it is a damage over time effect. And that damage should be able to be mitigated if you react fast enough. Here is how you can fix it to make it feel better for everyone.
So the spell is basically a BIG damage over SHORT time effect. Once you are within range, she can cast but even if you leave that range while the spell is being cast you still take full damage. This feels bad... really bad. If you move out of range, the remaining missiles shouldn't hit you.
How this SHOULD work is that in order for Kai'sa to do full damage she will need to stay inside the Icathian Rain range during the entire duration of the cast. Any missile that was fired after she moves out of range should simply NOT HIT. The missiles should have a max range that they can travel. If Kai'sa wants to do full damage, she would need to commit to staying in range for the duration of the cast. A person should be able to dash or flash out of Icathian Rain's range during the spell cast. If Kai'Sa wants to do full damage she would need to follow flash or use her Supercharge to stay in range.
This way you have counter-play on top of counter-play. Isn't this the type of shit Riot loves?
The amount of guaranteed damage from Icathian Rain just feels bad and since it takes a second to put all the damage down, you should have a second to react. The amount of damage you are able to mitigate will depend on how quickly you react and what tools you have available to get out of range. This sounds way more fun than how it's currently setup.