Has everyone forgotten the art of creating or using performance profiling tools
post
Never understood the idea of, "if it needs to be fast, that part is in a faster language." Why not just use the faster language? Most of the libraries that have Python wrappers also come with c headers. Sometimes you wind up in the reverse situation where your dependency doesn't have a Python wrapper and now you're stuck managing silly boilerplate. Python just feels like an extra thing in the way to wrangle for little gain.
Also, hotspot-only optimization doesn't seem to have held up in reality, right? We can all agree that software is getting slower every day. The infrastructure is slow.
All feels like cope, through and through.
All programming is a trade-off between what's easy for the developer and what's fast for the machine.
Sure, you could theoretically write everything in Assembly and it'd be blazingly fast; but unless you're a particularly elite developer, that process would take forever for any program more complicated than a calculator, the result would be super buggy, and those bugs would be very hard to fix.
On the other end of the spectrum, you might imagine a natural language machine learning compiler which translated directly from human prose to binary. It would be dead simple to "program" anything in; but such an application would probably be unbelievably slow, and again, have a staggering number of difficult-to-fix bugs.
Somewhere in the middle are modern programming languages that abstract away all of the stuff that's necessary but hard for humans to wrap their minds around (memory management, API interaction, network stack, etc) and let the human write clear and unambitious code to handle all of the business logic and displaying it to the user. The result won't run as blisteringly fast as writing directly in Assembly, and it won't be as quick to market as Vibes-to-ASM, but the application will be developed in a reasonable amount of time and be reasonably easy to maintain.
Why not just use the faster language?
Because the other language is more developer-friendly or faster to write/read even if slower to execute. Some code doesn't need to be fast because it is always waiting on other things anyway.
Some languages effectively prevent whole classes of runtime failures / bad code. But, they do that by doing more which takes longer, at compile-time, run time, or both.
Some programs can't avoid graph-structured data dependencies. In that case, the simplest solution is a garbage collector, and one built into the runtime is capable of being more efficient (and less buggy) than one written for a specific program.
Anyway, there are many factors that determine which language is "best", and not all of them are objective. That's why we don't just use the fastest language. (Currently, I do all my coding in either Idris or Purescript, but I'm also unemployed and just programing for myself, for now.)
IMHO it's easier to quickly iterate with Python than with C++. If you aren't even sure what you want and you're just writing stuff, then Python is great. Is "4" true? Ofc it is.
I worked at a place where a site had to show the number of members, but for some reason it didn't show the correct number. The reason was the query to the database was wrong, I corrected it in five minutes and the boss told me to put numbers manually, increasing them until we could display the correct query result because apparently the problem was there a long time before I started working there and it would make a bad image for the company if it was so easily corrected.
On other project a friend of mine told me about things were so slow and getting slower with time. Database didn't have keys, indexes, nothing you should expect a relational database to have.
I haven't worked in a while because of my health issues and I can assure you I'd be happier getting paid for watering plants than working with computers.
I'd add that most of the time nobody actually measured anything. at work we inherit slow sites all the time and the first guess is always "the stack", then you profile it and it's one query running per item in a loop or something like that. I had the same with my own side project, was sure I needed a bigger server and after measuring the problem was somewhere else completely. measuring first is boring but it's cheaper than any rewrite I guess
Profiling complex systems can be quite difficult. That said, I believe in the adage "Profile twice, optimize once.", adapted from the old carpenters adage of "Measure twice, cut once."
Slowness is rarely essential, but I can be far away from where you expect it.
That's what Open Telemetry is for, honestly, specifically tracing. I'm in the process of adding tracing for all of our web handlers, http client calls, database transactions/individual queries, and asynchronous jobs. Should give us a pretty good view into the performance of the system, and allow us to monitor for things like performance regressions without any dedicated profiling needed (though we certainly could add additional profiling information through Otel down the road). Since I know you're a Haskeller (recognize your handle), the hs-opentelemetryecosystem is quite solid and has integrations with a lot of libraries in the wider ecosystem, like wai, persistent, postgresql-simple, conduit, and a bunch more. I haven't needed to write too much of our own instrumentation, aside from our job system since that's a bespoke system.
Yeah we used hs-opentelemetry at my last paid position. The company was an MS shop, tho so we were using ODBC to connect to MS SQL Server which doesn't have great in-process profiling. But, at least we could figure out what quer(y/ies) were running long and then we could use the MS tools to look at the plan and fiddle with that. It was also a small org, so it's not like there was a lot of internal services in one flow -- basically everything either stayed in the Web Yesod app or in the IVR app. All our profiling could have been instrumented process profiling instead of having to use otel at all.
So, I haven't seen how easy/difficult it is to actually use otel across multiple internal service calls to satisfy one external request.
Nah, software is slow because it runs a web engine for GUI.
Noticeable speed? Sure. But I am so pissed if a fucking todo list app is a 100mb download and consumed 300mb memory when running, all because this fucker bundles and loads a whole browser engine to render itself.
I therefore do think tech stack matters as well.
Web development and using browser renderers for GUIs is one of my most hated aspects of modern software. 🤢
top 50 comments