The browser opens a popup asking you if you want to grant access to a specific device for a specific website, it's not like random websites can just run adb commands on your phone
It's only sorta ok right now because nothing uses it normally. If it were used mainstream for some legitimate purpose, that permission dialog would get ignored, and it'd become a security risk. USB isn't something web needs to touch unless you really know what you're doing.
Well it's a stand-alone program too, not just an extension. I kinda wish extensions could act as full programs too but computers need better sandboxing.
Edit: Wait, no we didn't. Chrome added WebUSB support after all. Wtf I'm disabling that