Since CloudFlare can make sites listening exclusively on IPv6 accessible to people with a IPv4-only connection, I decided to remove the A records altogether and let CloudFlare connect with my server via IPv6 exclusively.

This seems like a really good practice. Here’s why. For one, I don’t even need to assign a public IPv4 for the server at all (which some hosts that’s saving money). The other benefit is that it, arguably, somewhat increases security, by making it harder to find out the real IP of the backend server. Most evil-bots do bruteforce scans via IPv4 nowadays, only because there are only so many combinations, and trying them all seems quite plausible. With the massive IPv6 pool, it effectively decreases the possibility of finding out and hitting the right IPv6 to attack. It’s simply about the chances of finding out the real IP – since you’re routing all traffic via IPv6, nobody should, in theory, find out the real IPv6 of your backend. Even though it is “security by obscurity”, it gives measurable benefits and in effect is an extra layer of defense. Don’t treat it as the only one, though.

One more tip: Don’t decrease the entropy of finding out your real IPv6 by only using the last bits and leaving the middle part with zeros. Make a truly random subnet and if you can, randomize the last segment (don’t use sequential mapping too). Don’t use word-based IPv6 addresses either, since they’re too cool and seem to be the first thing to try by potential evil IPv6-enabled scanners. Means unfortunately no dead:dad:beef:cafe in your address :-)

Note: CloudFlare has a bug with internal routing, making it impossible to use CNAME records pointing to domains which only define AAAA records that are CloudFlare enabled. When you only have a CNAME pointing to a AAAA record, you’ll get CloudFlare Error 1000: DNS points to prohibited IP.
It seems to me this is a bug – CloudFlare is trying to route all CNAME record only to A records and doesn’t even try to use available AAAA records.
The workaround is to replace all CNAMEs for AAAA records, avoiding any CNAME for CF-enabled subdomains, but that makes it tiresome to update a record in the future (you’ll have to update all the subdomain records). I’ve reported that bug to CloudFlare.