Opaqueness and the Illusion of Greatness

When you don't know how things work, you image them as how you would want them to be, or according to an impression based on a random fact or advertising.

There is this old joke that goes like this:

When I was a child, I slept well since I knew someone was guarding me.
When I was in the army, I didn't sleep since I was the one guarding.
When I left the army, I didn't sleep since I knew who was guarding me.

If you served in the Army, I presume that you already know the joke and/or can relate to how sad it really is.

"What has this got to do with anything?" you wonder? When using open-source vs. closed source software, this is sometimes the case.

I got it when I was reading Jay Pipe's book "Pro MySQL". He mentions there that although the manual describes different parts of MySQL as different and encapsulated components, working beautifully together - in reality, it's all just a big pile of code, tightly coupled together and not always as modular as you'd like to imagine. He also tells of parts of the engine that were apparently written by different teams with different standards, where the code just looks and feels very different from the rest of the system. I didn't dig into the source code myself to this depth, but I believe him.
If you have worked before on the source code of an open source project, this is probably not a surprise for you - but for me it was a sort of a moment of re-enlightenment. The ones writing the database are people just like you and me, and they even write bad code sometimes. Only in MySQL, it's bad code anyone curious enough can see.

I wasn't used to this type of thinking for years, having grown up in the Microsoft closed-source ecosystem. I had no one from SQL Server come to me and tell me that they did some stupid mistakes here and there, and that the code is a mess. Yeah, there were Service Packs change logs with lists of fixed bugs, but I doubt someone in Microsoft would release a statement in the change log that tells how the sort sometimes didn't work (famous bug in InnoDB a while back).
The solid UI, the great documentation and the overall behavior of the database does its best to hide the implementation details from you. Since it was solid on the outside, I assumed it was solid on the inside. It was sort of anti-FUD. I wanted to believe it's as elegant inside as it seems outside.

I said "re-enlightenment" before, since this kind of realization actually did happen to me before, but I didn't let it sink in completely.
I've been working a lot with SQL Server in the past, and if you are familiar with it as well, you know that you can access the source code of all the internal system stored procedures. There are a lot of them, and usually you don't need to dig into their code, unless you're using some cool undocumented trick or trying to find out if something is a bug. I can tell you one thing - much of the code in those stored procedures is extremely ugly. No proper casing/indentation, hard-coded magic numbers everywhere... Maybe later I'll dig up one of these gems to show to the world. I mean yeah, it works, but still... Yuck.

Thing is, this anti-FUD was so strong, I didn't want to believe that it's that ugly on the inside - so I left my "beliefs" then as they were.

Conclusions:

1. Don't stay in the army.
2. Bad programmers can write both open and closed source programs.