How slow should you lose weight?

I know that based on Matt’s writings that, after refeeding, fat should decrease slowly. I’m wondering now, though- how fast is too fast? And does it coming off fast matter if you still have other markers of good metabolism- warm hands and feet, good digestion, sleeping well, etc? I don’t count calories and I’m just trying to start working out again to 1) feel strong, flexible, etc. and 2) help with body composition. Thoughts?