Main problem is the weight difference. It takes a lot more to spin the stainless since it weighs twice as much. Stepping down 1" will help keep the holeshot you had and you'll lose very little top end since your bringing up RPM's. That is only if you have any margin to increase them.
The difference between aluminum and stainless is less than 1" pitch and 1or2 mph. It is very minimal. Aluminum is lighter, thicker and is easier to repair. Stainless is stronger thinner, more expensive to repair and faster by 1 or 2 mph.