Personally if it were me, I would wait on the dairy. It’s ok to go slow on these things, and you may be better off starting with grains first and see how you do before you add in tons of dairy. If wheat is an issue, maybe start with rice first and ramp up, then add oats, then white flour, then some whole wheat, etc.
For dairy, you can always start with ghee, then butter, then yogurt, then cheese, then whole milk, etc. Least likely to cause issues up to most likely to cause issues.
The good news is, it doesn’t sound like you were low-carb before, so weight gain may not be as big an issue, unless you were really low-cal or something.