The fact that the damage from your stolen API token is then mitigated if you've reduced its scope still leaves you in a worse position than if it had never been stored in the text file and never been stolen in the first place.
First, it's not a question if you have reduced it's permissions. With an api token you simply can't do a lot of things that you can with a password.
Second, you don't use api tokens as a hobby. You use them because you want to use a tool that needs to have access to your account. Either you use an api token that has a limited set of permissions, or your password that can do anything. Independently of that, it will be stored in a plain text file, because where in the heaven would it store it so that it does not need to prompt you for it every single time? Yes, there are a dozen secret store programs that could be used instead, but a lot of programs will not have support for every one of them. I fail to see that in case how a token with fewer permissions is worse than a password with all the permissions.