help with gypsum ceiling

Hi,I have a drywall ceiling and the light fixtures are recessed LED contraptions; when you remove them and reinstall them the gypsum around the whole is flaking.

WHat can I do to finish off the hole and avoid the flaking when the light fixture is removed and later reinstalled.

Any help is appreciated.

Thanks

ccuschieri