You do understand that the model weights and the context are not the same thing right? They operate completely differently and have different purposes.
Trying to change the model's behavior using instructions in the context is going to fail. That's like trying to change how a word processor works by typing in to the document. Sure, you can kind of get the formatting you want if you manhandle the data, but you haven't changed how the application works.