Upgrading to 32GB won't slow your boot times.

No.

Boot time is almost entirely a function of your storage and CPU, not how much RAM you have. Modern SSDs load your OS in seconds. Adding more RAM means the system might spend an extra fraction of a second initializing memory during POST, but you won’t notice it.

If you’re worried about boot times, upgrade your storage to an NVMe SSD if you haven’t already. That’s the lever that actually moves the needle. RAM capacity? Not a factor.