In the "Rate the add-on" popup you have three buttons:
"Review", "I'll review later" and "Sorry, I don't want". The last one will close that popup and it won't shown anymore (same of the first one). Only the "I'll review later" will show the popup again after some times.

I'm sorry you found the popup too annoying 😢