Modest Programming Challenge: Let's try to write software that sucks less
I've heard it, you've heard it, there's absolutely no news in saying "WOOOW computer hardware sure is impressive these days! You can put a super-computer in your pocket!"
Sadly, an observation not quite as common is "and yet everything kinda sucks to use." It's true that common smartphones are incredible machines by the standards of computer history and much more powerful than home desktop computers were in the 90s. But there's a growing sentiment amoungst many variety of whiney programmers— which I definitely am by the way— that our field squanders this potential.
Don't mind me, just gonna make a sweeping statement based mostly on gut feel and anecdote, no biggie
I'm going to make a somewhat bold statement: I personally do not see any reason for the average consumer, and even most enthusiasts, to have computer hardware seriously improve from this point. I even feel that had our industry tried harder, computers from ten years ago would have easily fufilled the majority of needs. There are definitely exceptions to this such as video editors, but again they are exceptions.
Conceptually, when you think about the size of 16GB of RAM.... is a lot of information. That's 4 billion 32 bit numbers. If a common user workload can seriously approach that amount of data I feel like something is up.
And this isn't just pointless complaining— although that is one of my favourite passtimes. No, this is quite impactful. The world is effectively wasting large amounts of energy, industrial capacity, rare minerals, consumers' time and money on what really feels like nothing; we upgrade our computers and yet we don't have a snappier experience than we did 10 years ago. They say we do this because hardware is cheap and programmer time is expensive, but I'm curious if the cost of that hardware really factors in all the lost-opportunity cost and negative externalities. We don't have an endless supply of minerals, and hell, we're even in a silicon chip shortage right now. Additionally, this is often a negative externality: the cost is on the user's hardware. It's easy to say it's cheap when you don't pay for it.
Now, I could just sit here and complain. After all, what's the fun of a blog if not engaging in grossly oversimplified criticisms. But I think if we want things to change, we need to participate in and foster a culture of trying. I'm not gonna say absolutely everything must be done in the most optimized way ever. But I think it'd be good to begin a culture of checking these things, paying attention to our software's resource usage and asking "Does that seem reasonable?". I think that mindset alone would go a long way. Here on this blog I'm going to call this "Modest Programming": simply asking yourself if your program is reasonable in what it asks of the user's computer, and really trying at all.
This idea is far from unique. We've got suckless, we've got permacomputing, we've got uxn. I could name more. I don't claim to have invented this notion, and I'm not pitching "Modest Programming" as the term that should be used around the net. It's just the wording I like, and so I'll use it on this blog.
But enough with that. It's time to practice what I preach. Emphasis on practice.
Recently I was talking in a discord server that I frequent, and commonly exchange information and opinions about deeply obscure japanese video games. In case you had not figured it out so far: yes, I am in fact a filthy nerd.
There's a question that gets asked there reasonably often, that has no simple answer, and it raised to my mind the thought of "Man, it'd be kind of nice to have a community wiki for some of that stuff." Now, obviously random free wiki hosts like Fandom™ (formerly Wikia®) exist, but personally, I like owning community spaces. And setting up your own sites is fun. So I found myself looking into what wiki software is out there, and how cheap I could reasonably run a small community wiki.
Sadly, I couldn't really get much of a solid answer on how low-spec existing wiki software went. I'm sure they're not insane but if I could get this thing running very reliably on a 512 MB RAM VPS I'd very much appreciate that. And looking around it felt like none of the many perfectly high quality solutions had really been paying attention to this particular point. There is DokuWiki and don't get me wrong. It looks pretty solid. It doesn't even use a DB which I think is a good choice for both being light-weight and simple; the qualities someone would want for a small community wiki. But again, I couldn't really find exact details on just how low I could turn this thing down, and how well it'd handle it. Generally using little memory is one thing, but can it resist collapse in the face of some reasonable traffic?
Now, I haven't done too much web server programming in a wee while, and I felt like I should get some practice. Also, I'd like to give myself a Modest Programming Challenge™. So, why not try making a modest wiki implementation? A wiki is not the most impressive thing ever, and so you may think that that makes them not a great example for Modest Programming. I disagree. I think because wikis are so simple in their general function it should be reasonable to run one on some innanely small devices.
I will also admit, and I'm sorry if there's any PHP fans reading this, but a very large portion of established wiki server options use PHP, and quite frankly. I'm just prejudiced against any software built on that technology. I don't want to run it, I don't want to administer it. I'm absolutely sure you can point to perfectly good examples of PHP server software. Even wikipedia is one of them. But I've read too many things to feel comfortable touching it. Forgive me for my paranoia.
The product of this will have reasonably little use case. DokuWiki is likely a generally good solution for the small community case, and obviously MediaWiki is extremely battle-tested. But this is partially for practice, fun, and there are a few things I'd like to do different.
Speaking of which, let's lay out some goals.
Goal 1: Should run on extremely cheap VPSs
The specific use case I'm targeting is moderately technically capable people who want to quickly set up a small community wiki at extremely low cost. This site is hosted on Neocities so you can probably guess that I'm somewhat opposed to the "BigTech" CorpoWeb™. I think if we want more community-owned spaces, we need to make them as cheap to run as possible.
Additionally, as I have already stated. A wiki really is an extremely simple app in terms of what it actually does, sure it's not technically a static site but it's very close. I feel it should be possible to run on pathetic hardware.
Goal 2: Should be very simple to administer
Again, community-owned spaces online run largely on volunteer labour so we need to respect that labour. I think the general set of things someone should need to learn to use this and administer it long term is:
- What a VPS is and how to set one up
- Some bash
- Maybe some SQL
- An extremely simple configuration setup for the wiki settings
The last one is fairly important. Most small communities don't really need complex access rules. They want a wiki that is easy to set up and then gets out of their way.
Goal 3: Should be very easy to modify
Now, this one is something that DokuWiki does not satisfy in my opinion. I am a big believer in simple software for technical users who can come to understand the entire thing in quite some detail. I use a custom modification of makesite to generate this blog (okay, it's not the most impressive blog, I will admit) and I love the solution. It's a dirt simple python script with a template that generates a static site. If it doesn't do something you want, you can make it do it because it's so simple that you can read through the entire thing and its templates in a lunch break. And honestly, for a reasonably technical person that might be a better solution for customizability sometimes. Contrast trying to make a complex piece of software that can do everything as long as you want to invest the time in figuring out how you make it do anything. Then you pray to god that it doesn't surprise you with anything weird at some point that you can't debug because it's far too complex to just solve trivially.
So that's a thing I want for this modest wiki. DokuWiki doesn't fit the bill because according to OpenHub it's 233K lines of code. That's not a tool you can understand entirely.
Goal 4: Should deliver trivially small payloads, fast
I'm going to layer more stereotypes on to myself here and say that the web is a goddamn mess. Pages are unreasonably huge for most purposes. Things take way too long to load. I use Google's Pagespeed Insights to evaluate performance of my blog and I'm happy that it's a solid 100. It could be even more streamlined; most of my website's by filesize is fonts when I could just use fonts that come with most operating systems. But it's still pretty good.
Goal 5: Put in a reasonable effort to make it reliable
So this one is probably the most iffy goal of the lot. But ideally, the wiki should be able to survive some surges and not get taken out by weird edge cases. Now, high-availability is an entire subfield of software engineering, and it's very hard. Realistically I'm not gonna achieve that much on this front; I don't want to put the expectation on users to set up several nodes in a redundant server network.
But I should pay attention and do some tests. Try to make sure that there aren't obvious early bottlenecks and areas where the memory usage could spike high and have a single bad request take down the server for everyone. I consider this a side-aspect of Modest Programming: puting in some effort to try to preemptively eliminate degenerate situations that will cause instability for ends users.
Goal 6: Blog about it, for fun
I want to promote this idea of even trying to make software reasonable in it's demands. Doing it in silence is well... obviously impossible. So I'm going to share the adventure here, and some of my decision making process. Perhaps some of you will like the idea of the Modest Programming Challenge and try to make some modest software of your own, perhaps you'll like to ask questions or propose alternatives to my decisions. If you do either, tell me about it. I'd love to hear from you, see what you do, and if you give me permission I'll mention it here in the blog.
If you want to follow along with this journey, remember to put my RSS feed into your newsreader. And yes that sentence SHOULD be the new "like and subscribe."
The next post will be on my early technology choices. I think they'll be a little surprising, because I won't be going with a low-level language. I know the software could be even more optimized if I did, but I feel there's real headroom for modesty from just trying at all, without having to change entire technology stacks. Anyway, thanks for reading and see you next time.