Germany for one is already a federal country. Meaning that while a lot of things are done by the federal government but quite a few state-internal things are done by the states themselves, such as education for one.

This could be done for the EU as well