Yes, unfortunately this is not possible. All my Add-on does is toggling a Firefox built-in setting and this always applies to all sites.