The current model of libraries and open-source development (topics which Randall has addressed extensively in the past) relies heavily on the free and continued dedication of unpaid hobbyists. Though some major projects such as Linux may be able to garner enough attention to build an organization around it, many smaller projects, which are in turn reused by larger projects, may only be maintained by one person, either the founder or another who has taken the torch. Maintaining libraries requires both extensive knowledge of the library itself as well as any use cases and the broader community around it, which usually is suited for maintainers who have spent years at the task, and thus cannot be easily replaced. Thus, there are many abandoned projects on the internet as people move on to greener pastures. Far from the days of backwards compatibility, that's usually not a problem, unless a project happens to be far down the dependency chain, such as illustrated in the example, in which case there may be a minor crisis down the road for both the developers and the users down the chain.
I worked for the Linux Foundation on the Core Infrastructure Initiative supporting OpenSSL and other projects. The one that scared me was Expat the XML parser maintained by two people on alternate Sunday afternoons assuming no other distractions. We did get funding for a test suite. Joe Biden was a supporter of LF and CII and was going to host a fund raiser for us at the White House until a perverse result.220.127.116.11 22:46, 17 August 2020 (UTC)
- Are you trying to tell me that Biden and Harris weren't for CALEA, DIETYBOUNCE, and similar backdoors just like all the feds? When will they discover how to stop sending money overseas? https://blog.risingstack.com/controlling-node-js-security-risk-npm-dependencies/ 18.104.22.168 07:37, 25 August 2020 (UTC)
In the explanation, is "far from the days of backwards compatibility" a reference to something specific? I thought quite a few things made today were still backwards compatible, or am I mistaken? Zowayix (talk) 18:26, 30 August 2020 (UTC)
Relevance of Imagemagick?
Could someone perhaps add to the explanation an explanation of how this applies to Imagemagick (as mentioned in the title text)? —22.214.171.124 22:58, 17 August 2020 (UTC)
- I don't use it myself, but it is a very versatile standalone utility that does a lot through command-line (batched) processing or can be accessed through actual API interface (I use GIMP tools that way, in automation, when not using it directly as a manual interface, but I understand there's a lot of love out there for IM). There's potentially untold uses for that, hidden in the background of other applications. If it disappeared or changed in just the wrong way, could perhaps half the CAPTCHA dialogues suddenly break? Could a self-driving car company find its vehicles are suddenly blind? We might suddenly have so many fewer Doge memes! (Wow! Much up-to-datedness! So topical!).
- In Randall's (or his characters') world, that is. In our world, I see someone mentioned Leftpad in the Explanation, which probably needs more Explanation (or else wikilinking) but is an interesting thing that actually happened in our world, albeit not quite armagg3don for society... 126.96.36.199 23:22, 17 August 2020 (UTC)
- Imagemagick is the de-facto standard for Image processing. Since the 90's engineers were either adding support for new formats to ImageMagick or adding new language bindings for ImageMagick. This resulted in a single library that is available on almost every server and desktop platform and can read and write almost every image format. Using imageMagick is sometimes unwieldly. e.g. on nodeJS it actually spawns a sub-process to run imagemagick. But it is still the de-facto (and the only practical) choice in most cases.--Deepjoy (talk) 00:24, 18 August 2020 (UTC)
- I would put emphasis on the "almost every image format" ... there are lot of alternative image libraries, but most only support handful of formats (often just jpeg, png and gif). Meanwhile, I suspect not even Gimp supports as many formats as ImageMagick ... and, of course, Gimp is not really usable as library OR for shelling-out. -- Hkmaly (talk) 23:43, 18 August 2020 (UTC)
- The massive reliance on ImageMagick was recognized in 2002 by the developers of GraphicsMagick who needed to guarantee a stable version of ImageMagick and created their own fork. So while almost everyone uses and depends on ImageMagick (or think they are using ImageMagick when they are actually using GM) there is an actively maintained alternative. -- 188.8.131.52 17:10, 21 August 2020 (UTC)
from the late 2010s onwards?
I'm pretty sure re-use and modularization was a thing long before then. Maybe it got more popular in the 2010s, but it's been around since at least the '70s.
- The ideal of reusable code libraries has been around for nearly ever, but except for some popular Fortran statistics libraries I don't think it achieved widespread achievement until much later, e.g. CPAN. Barmar (talk) 03:25, 18 August 2020 (UTC)p
The timezone database (https://en.wikipedia.org/wiki/Tz_database#History) has been around since 1986. libc in various forms has been around as long as C has. Reuse and modularity is a fundamental principle of software engineering, and not an invention of the last few years. I'd just remove any mention of date.
- I think it's relatively recent that you can delete a file from one Web server and everything on the internet breaks. Dependencies are one thing, dependency on live updated resources is new. Because it's rather a bad idea. Incidentally overall... I think today's comic needs to be explained slower. Most people in the world are very unfamiliar with these concepts. Although coronavirus responses have taught a lot of us about "supply chains" that put stuff into shops for us to buy. Robert Carnegie [email protected] 184.108.40.206 10:18, 18 August 2020 (UTC)
- While libc in various forms has been around as long as C has, it was never SINGULAR. Every version of C compiler had it's own version of C library maintained by different people. Even now there are alternatives to GNU libc. The timezone database might be better example. Also, reuse and modularity is fundamental principle, but reusing code maintained by someone else in project with bigger staff than that of such code is relatively recent. -- Hkmaly (talk) 23:48, 18 August 2020 (UTC)
German Television referencing this comic to illustrate the Log4j dependency (at around 1:11)
This has happened before
It may be worth mentioning a case where this actually happened, like https://www.theregister.com/2016/03/23/npm_left_pad_chaos/ 220.127.116.11 01:03, 18 August 2020 (UTC)
- That was only a problem for those who tried to compile against network versions, instead of having a local copy. One of the dumbest and laziest things you can do as a programmer. Not to mention that you could just copy the code directly into one of your files or just writing your own routine. SDSpivey (talk) 02:04, 20 August 2020 (UTC)
- Speaking as a SecDevOps person, another risky thing programmers do out of ignorance is host static local copies of code repositories without a good update and security review plan to make sure the static copy gets regular testing and updates as security and bugfixes are published to the source. Still another risk is writing your own library to reinvent the wheel and making the same mistakes the maintainer of the wheel solved six major versions ago. I would be careful throwing terms like "dumb" and "lazy" around. Every one of those solutions, including your proposals, *also* can be risky if implemented without proper expertise and forethought. There is no 'best' practice here, just risks and advantages that make it so that there is no single one-size-fits-all solution 18.104.22.168 13:30, 27 August 2020 (UTC)
One particularly big risk that instantly came to mind is the timezone database, which is maintained by volunteers yet underpins basically everything: https://en.wikipedia.org/wiki/Tz_database#Maintenance
I remember hearing about this a few years back at a Linux Foundation conference - the NTP daemon was underfunded (as I recall) and the one person maintaining it was struggling to pay bills. Losing NTP breaks an awful lot of things.... 22.214.171.124 19:48, 18 August 2020 (UTC)
- I see this was problem in 2016 ... I'm not able to find any update on the situation ... -- Hkmaly (talk) 00:10, 19 August 2020 (UTC)
- Nice long interview with Harlan Stenn, author/maintainer of NTP. RandalSchwartz (talk) 05:56, 19 August 2020 (UTC)
- I work with a E100k robot that keeps breaking on account of [Atomic Parsley]. Everyone is very amused at this Kev (talk) 13:32, 20 August 2020 (UTC)
Some random person in Nebraska
Is the reference to a random person in Nebraska totally arbitrary, or is it a reference to someone in particular?
Also, it would be good to have examples of heavily used projects with very small (especially one person) maintainer teams. OpenSSL definitely comes to mind, from what I have read. Stevage (talk) 01:49, 18 August 2020 (UTC)
Nebraska came up in 1667, "Algorithms" as well.126.96.36.199 02:22, 18 August 2020 (UTC)
- Nebraska is... Well, I'm sure some Nebraskonians might have a more fully-fleshed out and accurate opinion of its subtleties, depth of culture(s?) and Deity-given geographic artisanship but viewed from further afield it is one of the contenders for "miles and miles of not much going on", or similar, peopled by people that largely live within that promise.
- It may be just a meme of such a generality, as a brief look at a list of people from Nebraska tends to support the hypothesis that the ones who became significant (Astair, Brando, Carson...) probably did so only once they left.
- OTOH, there are (at least) four computing pioneers/developers mentioned among them, creator or authors of significant 'products', and maybe one of these matches the (intellectual) dependency meme quite well - other than being written in Massachusetts. Or this one, though that might have been LA-baked, maybe?
- I learnt some interesting things when investigating this issue, just now. Cheers! 188.8.131.52 09:54, 18 August 2020 (UTC)
- I feel like Nebraska is mentioned just because ot's the.most flyover-sounding flyover state name? Or is it actually home to some well known library maintainer? -- 184.108.40.206 (talk) (please sign your comments with ~~~~)
- Another good example might be left-pad. It actually caused a big issue in 2016 when the developer took it offline and a whole bunch of projects and websites broke. Numbermaniac (talk) 07:41, 22 August 2020 (UTC)
Microservices reference is not related to this comic, as ImageMagick is monolith application. Also microservices are way of operating and deploying web services, not utility apps.
220.127.116.11 07:56, 18 August 2020 (UTC)
- ImageMagick is a library. -- Hkmaly (talk) 23:50, 18 August 2020 (UTC)
The Thirty Million Line Problem
See The Thirty Million Line Problem. Randall's drawing looks like a house of cards on the verge of collapse. In the video, Casey talks about how the lack of a "hardware ISA" causes critical software (like OS'es and browsers) to bloat like crazy (a "hardware ISA" would be a standard for how hardware works, just like the x86 ISA is a standard for how an x86 CPU works, that both AMD and Intel agrees on). Also, he mentions how fragile and broken software is due to this "Thirty Million Line" bloat.
18.104.22.168 19:48, 18 August 2020 (UTC)
- Based on related discussion, that's a VERY bad video: he may have a point, but it takes VERY long time before he gets to it. I'm not going to watch it that long myself. -- Hkmaly (talk) 00:03, 19 August 2020 (UTC)
- This reminds me of that old joke: If carpenters built buildings the same way programmers made programs, the first woodpecker that came along would destroy civilization.
- I thought the drawing looks more like the Jenga game, except the components are not simple rectangles. Barmar (talk) 16:31, 20 August 2020 (UTC)
"Famous" Left Pad Incident
Log4j Zero-Day Vulnerability (CVE-2021-44228) Incident
On December 9, 2021, security researchers discovered a flaw in the code of a software library used for logging. The software library, Log4j, is built on a popular coding language, Java, that has widespread use in other software and applications used worldwide. This flaw in Log4j is estimated to be present in over 100 million instances globally. If exploited, could permit a remote attacker to execute arbitrary code on vulnerable systems.
There is an initiative by Eric Raymond targeted specifically to mitigate this problem.
Website: https://esr.gitlab.io/loadsharers/ — Smartchair (talk) 16:20, 19 August 2020 (UTC)
The Network Time Protocol is also a great example. --Slashme (talk) 21:50, 19 August 2020 (UTC)