It's the problem with YouTube injecting a ton of useless DOM elements which FadBlock has to go through. The lag stabilises after a while and YouTube starts working smoothly.

I am working on targeting the skip button more accurately to increase the performance. Still, it will be delayed a bit as there are higher-priority issues coming in the next update.