I've been gluten free for going on 9 years (celiac).
Not knowing your recipes, it's difficult to advise.
But as a general rule, crumbling is due to a lack of binders.
Commonly used binders include xanthan gum, guar gum, psyllium husk, chia seeds, flaxseed meal, and arrowroot.
Depending on the reason for the gluten-free diet, the gums (xanthan gum and guar gum) can be an issue. A lot of celiacs (myself included) can't tolerate much xanthan gum. I use it very sparingly.
Xanthan gum is one of the best binders, and the most controversial. It's gluten free, but some celiacs won't eat it as a matter of practice because wheat starch can be used in the fermentation process.
Chia and flaxseed meal can impart a gummy texture, so I only use them in combination with other binders and in small amounts.
Adding starches like tapioca starch and potato starch to your gluten free flour blend will also help with binding. But starches impart a styrofoam like texture. So it's important to use gluten free flours with flavor and texture. I like to blend brown rice, sorghum, and millet flours with tapioca and potato starches.
Is egg off the table? Eggs and egg whites are also binders.
In my gluten free pie dough, I use several binders, including pectin. I know gelatin is used by some gluten free baker's, but gelatin is derived from animals sources. As such, it's not suitable if you need to keep it vegan.
Puréed fruit and nut butters are also used as binders. The issue with these is fruit adds sweetness and moisture; nut butters add distinct flavors and oils. And of course there's the allergy component to nuts.
Over the years is used every binder I've listed here except gelatin. They all have there pros and cons. I found the key to a good gluten free product is using several binders in combination with a good gluten free flour blend.
Dairy is easily substituted with oil or butter substitute. When making the substitution, you have to reduce the amount of oil and butter substitute.
Aside from binders, the other issue with baking gluten-free is the slow hydration absorbency rate of gluten-free flours. More often than not gluten-free doughs and batters need to rest before baking. The rest allows the dough/batter to hydrate. Also, because of the low hydration rate, a bit more moisture and fat is usually needed.