The self-interaction of short electron bunches with their own radiation field can have a significant impact on the longitudinal beam dynamics in a storage ring. While higher bunch currents increase the power of the emitted coherent synchrotron radiation (CSR) which can be provided to dedicated experiments, it simultaneously amplifies the strength of the self-interaction. Eventually, this leads to the formation of dynamically changing micro-structures within the bunch and fluctuating CSR emission, a phenomenon generally known as micro-bunching or micro-wave instability. The underlying longitudinal dynamics can be simulated by solving the Vlasov-Fokker-Planck (VFP) equation, where the CSR self-interaction can be added as a perturbation to the Hamiltonian. As CSR is emitted at wavelengths corresponding to the spatial dimension of the emitter, these small structures can lead to an increased emission of CSR at frequencies up to the THz frequency range. In pursuing control, we thus consider a deliberate excitation as well as mitigation of the micro-bunching dynamics to support different operation modes. To address the CSR-induced perturbation at the timescale of its occurrence we use RF amplitude modulations to counteract or amplify its effects. The necessity of dynamic adjustments of the applied control signal naturally motivates the use of reinforcement learning (RL) methods. We demonstrate the control achievable with this approach in simulations and present first measurements at the KIT storage ring KARA (Karlsruhe Research Accelerator).