Well, that was just a quick summary of an idea. You can't just tell someone (or a an LLM) to think critically and expect it to work magically. However, you can follow certain guidelines to take a few steps in the right direction. There is a method to it you know. It starts with simple things like investigating whether or not a claim is supported by the evidence, are there unstated assumptions and so on. If you follow rules like this, you can avoid some of the simplest reasoning mistakes. Putting all of that into a prompt takes some time and effort. Doing it properly will probably result in a 500 line long manual of critical thinking.

I've done something similar with programming, and the LLM usually follows most of my style guide instructions reasonably well. It's not perfect though, and it will deviate from some rules no matter how hard you try. Also, it requires constant hand holding, because it's an LLM. Anyway, I don't expect a critical thinking agent to be a fool-proof solution. As long as it can avoid some of the stupidest reasoning mistakes, it could be a bit more useful than the default version, and might actually be helpful in troubleshooting.

If you have a list of commands to type in the terminal, but you need them only once, you aren’t going to make a bash script for that. If you think you’ll need to type that thing every week, making a script suddenly begins to make sense.

Same thing with agents. Do you think you’ll be asking these kinds of questions again in the future? If so, making an agent for it could make sense.

Googling the solution yourself can honestly be such a pain in the ass for some problems. Search engines have become so shite. You'll find some stack overflow question from 8 years ago that's no longer relevant, and in the end you might have to consult documentation directly which can take many minutes or hours. I reaaaally hate to say it but LLMs are actually useful for this usecase