I can comment out rules to remove them from the firewall custom rules and restart the firewall. This works just fine. However, when I add a custom rule and restart the firewall, I can see it in the firewall status under the right table, but it doesn't actually do anything until I restart the whole router.
Did you click "Save & Apply"?
I didn't before, but that's because there is no Save & Apply button on that tab, I had to go to the firewall settings all the way to the left, wait for that page to load, then click Save & Apply. Due to your question, I tried just that and I can confirm that I still had to reboot the router in order to apply the rules.
I am not able to replicate this issue. What release of libreCMC are you using?
I'm using Think Penguin's latest OEM version.