The reason why the extension has "access all website information" because it works on any website that has embed YouTube videos.
https://support.google.com/youtube/answer/171780
https://developers.google.com/youtube/iframe_api_reference

Also, the source code is open source. You can look at the code and report bugs here
https://github.com/Dosx001/YouTubeUtils/