Patagonia's baggies are the only answer.

Yes. Patagonia’s Board Shorts (the classic baggies or their newer Wavefarer line) check both boxes without pretending to be something they’re not.

They use recycled nylon (made from fishing nets and waste), the back pocket zips and won’t lose your keys, and they dry fast. Most “recycled” board shorts from fast-fashion brands use thin fabric that pills or fades in a season—Patagonia’s stuff holds up for years.

The secure pocket situation is simple: a single zippered back pocket that’s big enough for a phone and a wallet. You don’t want cargo pockets slapping around in the water.

One caveat: if you want a true technical short for surfing, get the Wavefarer. If you’re just wearing them to the beach or pool, the baggies are fine. Both are made from recycled materials and both have a pocket that won’t betray you.

I’ve had a pair of baggies for six years. The elastic is looser than it was, but they still work. That’s more than I can say for the three pairs of Target shorts I went through in that time.