Can't get different state background for MaterialButtonToggleGroup

Question

I'm using Material 3 ButtonToggleGroup, I want to have a different background when a button is selected. However, If I don't add any attributes to my style applied to MaterialButton, I get the background set to colorPrimary when a button is selected but when it's not it's transparent.

When I add a backgroundTint attribute, it always has the same background color, no matter if the button is selected.