you are viewing a single comment's thread
view the rest of the comments
[–] [S] 5 points 2 days ago (6 children)

My experience is completely different. I've been able to implement a ton of ideas I've had for years using them that I simply did not have the energy to do before. The speed of development is significantly faster when you use them properly.

I've also haven't found no problems with maintaining projects where I use LLMs. I spend the time to design the architecture and structure of the project. This acts as the scaffold, and the LLM simply fills in the implementation details. If anything, I find the code ends up being cleaner because it's now much cheaper to do refactors, and to experiment before you settle on a solution.

The pricing is only an issue for American models. DeepSeek is already making profit, and there is no reason for them to raise prices because they never took on huge amounts of debt they need to pay off while their model continues to become more efficient. On top of that, local models are viable as well now. Qwen 3.8 27b is roughly equivalent to Opus 4.6 from less than a year ago. I fully expect that the trend of models getting smaller and more capable will continue.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 3 points 2 days ago (5 children)

    Local models are the ones that have appealed to me the most, but even with 128GB of RAM I can't run the models I wanted to run like the latest qwen model, which was a disappointment for me. A crossword puzzle generator has been my metric for testing them and I've never seen one able to create it from scratch. They've gotten much better in the past couple years, but are still incapable of what I consider a pretty basic challenge.

    I'll probably get the itch in a month or two and try the latest local models requiring no more than 128GB RAM and 24GB GPU produced by China.

    My coworkers all use anthropic and openai based models so they will absolutely get trashed when the capital subsidies stop. Definitely you're right that the Chinese models aren't bound by the same financial scam.

  • source
  • parent
  • hideshow 5 child comments
  • [–] [S] 5 points 2 days ago (4 children)

    The models run on the GPU, so the amount of RAM isn't really important here. What matters is how much VRAM you have. Also, the process isn't to try and make one shot programs using the model to make all the decisions. You lay out how you want to structure things, make a plan, and then have the model implement it incrementally and refine the result. It also helps to break down the problem into small tasks since that helps control the context.

    With 24GB VRAM you can only run really small models that aren't terribly capable. You could technically run Qwen3.8 27B at Q6_K, but that's a really tight fit meaning it's going to have a very small context. 32GB would a be a more comfortable fit there.

    And yeah, Anthropic/OpenAI don't have a business model in my opinion, so it is only a matter of time till the bubble pops.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 2 days ago (3 children)

    I don't think a crossword puzzle generator requires all that much context. Maybe they will get there at some point. They are still consistently confidently wrong, which makes them feel super untrustworthy to me. If a coworker said to me "oh, you're right, this fixes it" 4 times in a row without fixing anything, I would not trust that coworker. LLMs do the same thing.

    You can't even run them if you don't have enough system ram, but yeah, graphics vram is for sure very important.

  • source
  • parent
  • hideshow 3 child comments
  • [–] [S] 2 points 2 days ago* (2 children)

    LLM assisted development has nothing to do with trust at all. Think of it as a genetic algorithm. The model makes a rough approximation as a first pass, then it runs tests or specs, gets feedback, and iterates on it. And each iteration gets closer to what you specified. The better the specification that you provide is, the better the end result will be.

    Meanwhile, context isn't some abstract thing. It's the amount of tokens the inference engine can hold in memory. With the memory constraints you have, it's going to be tiny, which means the model isn't able to hold the whole solution in the context even for a small thing like a puzzle generator which will be thousands of lines of code.

    Good news is that pretty much any problem can be broken down into a series of steps. And then if you structure things well, each step can be treated as its own isolated problem which doesn't requite much context. With the example of a crossword puzzle, what you'd want to do would be to create an outline of the shape of the solution yourself, how the pieces fit together, and what the end result looks like, create tests for the functionality, maybe even files with stubs for the functions you want if you're working with a small model. Then you run an agentic harness like opencode where the model can iterate on the solution, run tests, and get feedback on what it's doing wrong. You create the scaffolding for the solution and you own the contract, while the model fills in the details for it.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 days ago (1 child)

    Trust is relevant in the same way it's relevant with old school blueprints, dependencies, and with other automation.

    In my experience the LLMs don't consistently get closer.

    A puzzle generator doesn't need to be thousands of lines of code by any stretch.

  • source
  • parent
  • hideshow 1 child comment
  • [–] [S] 2 points 2 days ago

    Not really because trust was never how we ensured software quality in the first place. What matters is the process, doing things like code reviews, having a good test harness, coming up with specifications, doing manual testing, and so on. LLMs don't really change anything here.

    I'd also love to hear how you actually use LLMs and what your workflow is, because they absolutely do get closer when you set up constraints correctly and run them in the agentic loop in the harness. I also don't know what you refer to by puzzle generator here, clearly you mean something simpler than I assumed. If you want to write out what the specific requirements are, I can show you what Qwen 3.8 comes up with on my machine for them.

  • source
  • parent