IT#14 Economy of Complexity

The Paperwork Reduction Act (PRA) is a United States federal law enacted in 1980 designed to reduce the total amount of paperwork burden the federal government imposes on private businesses and citizens.

United States Federal Law enacted by the 96th United States Congress. Effective December 11, 1980.

#ITManagement #Management #Economy #ParasiticLoops #Complexity #TechnicalDebt

This article explains the phenomenon of Economy of Complexity, its kinds and when does it become really bad.

What is Economy of Complexity?

What is this, Economy of Complexity? To make it short, the economy of complexity appears when a significant cost for a customer comes not from the materials and labor costs, but rather from the artificially raised beyond measure complexity of the process, service or a product.

The classic example of the economy of complexity is bureaucracy. Imagine that you live a couple of centuries earlier than our time and come to some small European city to an insignificant city clerk with some need. Maybe you need to register buying a house or notarize a document. What do you bring with you besides documents? Right. Ancient Romans called it the “praemium”, we call it simply “bribe”.

However, from the point of view of city clerks. Just one bribe is too small. Also, they will try to support each other and make more clerks to earn money. So, the clerk, who should notarize your document, will for sure need a copy from the archive. And then the archivist will need, say, a letter from the head of the city guards confirming that you are a good citizen. Of course, the head of the city guards does not know you, so he will need confirmation from the gaffer who patrols your area... Long story short:

Go to the second 168 (2 min 48 sec) or use the link to YouTube.

Linguists noticed that the more important thing is for some people, the more synonyms they have for it. Here is how bribe is called around the world:

àbẹ̀tẹ́lẹ̀, bertîldayin, bestechen, Bestiechung, bestikke, breab, brìb, chiokomuhomwe, chiphuphu, cin hanci, corruzzione, eroskeria, fa'ato'aga, iri ngarị, isinyobo, kipe, kolikoly, kukuli, kyšį, laaluush, lahjoa, llwgrwobrwyo, megvesztegetni, menyuap, mită, mito, mua chuộc, muta, Mútur, nyogok, omkeapje, omkoping, para, podkupnina, podplatiť, pora berish, pot-de-vin, przekupić, rasuah, rushwa, rüşvət, ruswa, ryshfet, suborno, suhol, tangente, tixħima, tjotjo, ukufumbathisa, úplatek, whakapati, xiab, δωροδοκία , взятка , мито, пара, подкуп, пора додан, поткуп, ришвәт, хабар, хээл хахууль, ქრთამი, կաշառք, כאַבאַר, שוחד, پارا, رشوة , رشوت, رشوه دادن, ጉቦ, घुस, रिश्वत, लाच, ঘুষ, ਰਿਸ਼ਵਤ, લાંચ, ଲାଞ୍ଚ, கையூட்டு, లంచం, ಲಂಚ, കൈക്കൂലി, අල්ලස්, สินบน, ໃຫ້ສິນບົນ, လာဘ်ထိုး, សំណូក, 뇌물, 賄賂, 贿赂.

Notice that it reads the same in Chinese Traditional and Chinese Simplified: 贿赂/賄賂 - “huìlù” (in case you plan to visit China). And it exists even in Esperanto: “subaĉeti”! Clearly bribes come very deep in the human civilization, and where do they happen? When someone has some arbitrary power over someone else, like in a government bureaucracy.

However, even if they don’t take bribes, they need to look busy!

• This gives clerks job security: everyone sees that you are busy and necessary.
• This gives city clerks' managers reasons to ask for more funds and more people, both convertible to their personal income.

And I don’t say that the people involved in that are trying to increase expenses of the city in a bad faith. Most likely nearly all of them are convinced that all those papers and documents are necessary. But it does not matter what they believe in. What matters is that we have here a useless machine that wants to procreate and expand itself like a cancer and influences the substrate (the city) to do so. And it can do so in part by creating more complicated instructions, requiring more paperwork, involving more of its people in every process. By increasing the complexity of the service.

You can read a more common case in the upcoming “Self-Procreating Parasitic Processes” article. You can also read why city clerks are more interested in that goal than in the well-being of city council visitors in another upcoming “Large Systems” article. But back to the economy of complexity.

Again, the economy of complexity exists when a significant cost for a customer comes not from the materials and labor costs, but rather from the artificially raised beyond measure complexity of the process, service or a product.

And in most cases that complexity is supported and cultivated by people who have an interest in growing that complexity. What’s also important is that formal managers, leaders or just boss has limited capability to counteract it as well as little interest to do so.

Unfortunately, medieval small-town councils are not the only ground where an economy of complexity can grow. Let's now forget about the example of bureaucracy and get to the modern knowledge economy and IT. Which is a very rich soils for this generally negative phenomenon.

Here are two examples, which I observed firsthand. I normally hate case studies, since most of them seem to study a case of a cockroach who survived bug exterminations since it accidentally fall into a deep crack in the floor near wall.

But these two cases are sensible, systematic and may be useful for you. While I worked for more than a decade in both companies, no internal information will be exposed, only what you can find on Internet and hear from veterans of both companies.

External Economy of Complexity: Microsoft

We did not know how to make easy to use and easy to understand APIs when CP/M, MS-DOS and Later Windows was first shipped. We did know something about UI design, thanks to IBM SAA (Systems Application Architecture) standard and especially its two parts CUA (Common User Access) and CUI (Windows Consistent User Interface standard).

So, UI was fairly good. APIs were not. I remember Bill Gates proudly saying that Microsoft has a secret technology which makes software development for Windows much easier. Today we all know this technology as COM/DCOM, and most engineers prefer to avoid it, if they can.

Another legendary complicated Windows API was CryptoAPI. When the speech on Crypto API at Microsoft DevCon 2006 started with words “We made CryptoAPI much easier...”, it was met with ovations. Believe me, I know it was and still bad. In Windows Home Server I did a unique client-server architecture with generated client and server certificates and two-way authenticated TLS connection. Often times, I felt like a hunter-gatherer shaman dancing around a tribal fire with a tambourine.

And you know what? Sometimes tambourine DID help. I think this is the most important thing you need to know about the complexity of Windows CryptoAPI.

Nevertheless, the initial complexity of development for Windows was not a result of any ill intent, it was a natural result of the state of software engineering at that time. But the result was still too complicated and too expensive to use for business customers, who heavily relied on Windows custom add-ons.

However, with the factual monopoly of Windows on the market the outcome turned out positive for Microsoft. The complexity of developing for Windows created a whole business eco-sphere of Windows consultants and VARs (Value-Added Resellers). The existence of these businesses and well-being of people involved in these services heavily depended on the existence of Microsoft and Windows.

It did not help it much with the anti-monopoly suit, and on Friday, Nov. 5, 1999 judge Thomas Penfield Jackson declared Microsoft a monopoly. I vividly remember that since at that time my MSFT stock options went underwater.

Still, that business eco-sphere was an important factor forming the public opinion. It also created a lot of people who depended on Microsoft for their well-being. This was good for the economy, and not only in the United States. Of course, business customers in the end paid high premiums, but ROI on these projects and services was normally positive and, as I said before, the complexity originally did not come from ill intentions. And once the business eco-sphere around Microsoft has formed, it was not a real priority for the company anymore to simplify usage of its products.

That said, this would not happen if Windows had a competitor OS simpler to develop for it. However, the only real competitor, Apple MacOS was not better, and it had much less accessible developer tools. Linux was still in pumpers, SunOS was heavily overpriced and targeted to very expensive by modern standard hardware as well as generic Unix. Xenix, which I happily used on PCs in the late 80s to teach students C and C++, was owned by Microsoft, which did not have much interest in pushing this OS onto the market. So, in the end, business customers did not have much choice other than Windows. 

And so, the external economy of complexity worked. Microsoft was making around $1B per month and as rumors say, on one of the company all-hands Steve Ballmer said “It’s better than selling drugs.”

Internal Economy of Complexity: Google

However, the economy of complexity may work another way, and then it’s definitely not good. We will speak about the economy of complexity when the cost falls not on external customers but on the internal development. Basically, the company punishes itself and does not know what to do about it.

Yes, I speak about Google. It has (or at least had until lately) a peculiar promo system, when promo decisions are made by committees of senior engineers, not management. Until lately these committees contained engineers from different PAs (Product Areas). For example, a promo of an engineer in maps could go for consideration by engineers in, say, YouTube. Naturally, members of such a promo committee did not have the slightest idea how important the contribution of the candidate is. All they had were reviews by managers and peers of the candidate from his and partner teams, which, naturally, always had a lot of exclamation marks and panegyrics to the candidate’s work. Which, again naturally, was not very informative.

Measuring the candidate performance in dollars added to the bottom line of the company is highly problematic, especially for companies of the size of Microsoft or Google. By the way, if you know a way to do that, please, let me know.

So, the only real ground of promotion left was the complexity of what the candidate did. Read that sentence again, please. Do you hear it? Complexity. That’s it. You get what you pay for.

I remember once at Microsoft we had a director-level manager who thought he was very smart. He postulated that now perf will be measured. Testers will be evaluated by the number of bugs they opened, and developers by the number of bugs they fixed. And guess what? We demonstrated a really good teamwork. That period we had a record number of bugs opened and fixed. “Could you add a comma to that comment in the code, please?” - “Sure, done! Thanks for noticing!!!” It took that manager a while to stop that perpetuum mobile.

Basically, in the Knowledge Economy no matter what metric you will use to measure the performance of your employees, some of them will figure out how to use it to their advantage and disadvantage of the company. And, by the way, these are those whom you will promote. After all, in the Knowledge Economy you have to hire those who are smarter than you. So, what do you expect? However, this is a subject to other articles, already published “Corporate Parasites” and upcoming “Why Perf is Bad for Your Company?”

This is how even a good company may end up with the internal economy of complexity. You promote for complexity. You end up with tons of highly complicated and overengineered internal systems. And they have to work together. What does it do to your internal costs?

Maybe that’s what that self-important dude from the City of London should had write about, instead of asking to fire a lot of people.

Anyway, this is the case of the internal economy of complexity enforced and cultivated by the company’s perf system.