Difference between revisions of "1654: Universal Install Script"
(→Explanation) |
Chrullrich (talk | contribs) ((too much) useful information) |
||
Line 19: | Line 19: | ||
One of the more subtle jokes in the comic is the inclusion of <code>apt-get</code> and <code>sudo apt-get</code> in the same script. In most cases this would be redundant as the <code>sudo</code> command is just to add admin permissions. This could be a reference to a joke in the Linux community about forgetting to include the sudo command. An example of this joke being used elsewhere was a [https://twitter.com/liamosaur/status/506975850596536320 viral tweet] that showed a workaround for the issue. Sudo has also been used both by [[Randall]] in [[149: Sandwich]] and by Bill Amend to force Randall to let him appear on xkcd with [[824: Guest Week: Bill Amend (FoxTrot)]]. | One of the more subtle jokes in the comic is the inclusion of <code>apt-get</code> and <code>sudo apt-get</code> in the same script. In most cases this would be redundant as the <code>sudo</code> command is just to add admin permissions. This could be a reference to a joke in the Linux community about forgetting to include the sudo command. An example of this joke being used elsewhere was a [https://twitter.com/liamosaur/status/506975850596536320 viral tweet] that showed a workaround for the issue. Sudo has also been used both by [[Randall]] in [[149: Sandwich]] and by Bill Amend to force Randall to let him appear on xkcd with [[824: Guest Week: Bill Amend (FoxTrot)]]. | ||
+ | |||
+ | Another explanation for this could be that plain "apt-get" is for Debian, while Ubuntu etc. use sudo. | ||
The tool <code>curl</code> downloads files from the network (e.g., the Internet). Used like <code>curl http://xkcd.com/</code> it downloads the xkcd main page and displays the HTML source code. The pipe <code>|</code> in the script attaches the output of the command before the pipe to the input of the command after the pipe. Both commands are executed concurrently. <code>bash</code> is a popular shell for Unix-like operating systems. The line <code>curl "$1" | bash</code> tries to download a file from the network and to execute the download directly. <!--- Although this is a common practice for conveniently installing software, it is considered extremely insecure and should never be done. [ed. note: there's no reason this cannot be secure, especially if HTTPS with validated certificates is used, from a trusted domain which utilizes DNSSEC] --> | The tool <code>curl</code> downloads files from the network (e.g., the Internet). Used like <code>curl http://xkcd.com/</code> it downloads the xkcd main page and displays the HTML source code. The pipe <code>|</code> in the script attaches the output of the command before the pipe to the input of the command after the pipe. Both commands are executed concurrently. <code>bash</code> is a popular shell for Unix-like operating systems. The line <code>curl "$1" | bash</code> tries to download a file from the network and to execute the download directly. <!--- Although this is a common practice for conveniently installing software, it is considered extremely insecure and should never be done. [ed. note: there's no reason this cannot be secure, especially if HTTPS with validated certificates is used, from a trusted domain which utilizes DNSSEC] --> | ||
Line 24: | Line 26: | ||
The use of & at the end of each line causes the shell interpreter to execute the commands in parallel (asynchronously) instead of sequentially. Even if single commands fail, the rest of them will be executed. Note this is even the case for the final commands which attempt to change to the installed package, probably the only reason why this may not work completely for packages that do need compiling after being downloaded. (However, just running this script again would probably do the trick.) | The use of & at the end of each line causes the shell interpreter to execute the commands in parallel (asynchronously) instead of sequentially. Even if single commands fail, the rest of them will be executed. Note this is even the case for the final commands which attempt to change to the installed package, probably the only reason why this may not work completely for packages that do need compiling after being downloaded. (However, just running this script again would probably do the trick.) | ||
− | The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running could simplify things further. <code>2>/dev/null</code> redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it. | + | There appears to be a bug with the & at the end of the "git clone" line; since a git repository typically contains program source code, not executables, it may have been intended to retrieve the source code with git and then compile and install the program in the next line. In this case, the single & should be replaced with &&, an operator that will run the second command only if the first one has completed successfully. This plays into a second bug on the "configure" line, where the placement of the & means that only the "make install" command will be run asynchronously after the "configure" and "make" steps have finished in sequence. To make success as likely as possible, the two lines should be like this: |
+ | |||
+ | git clone <nowiki>https://github.com/</nowiki>"$1"/"$1" && (cd "$1"; ./configure; make; make install) & | ||
+ | |||
+ | The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. Since all the commands come from different operating systems, versions, or distributions, it is not very likely that more than one will work (with the exception of pip/easy_install and the two forms of apt-get) or even exist on the same system. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running, by making them read input from another program that writes a (nearly) endless stream of "y"s, could simplify things further. <code>2>/dev/null</code> redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it. | ||
==Transcript== | ==Transcript== |
Revision as of 14:09, 11 March 2016
Explanation
![]() |
This explanation may be incomplete or incorrect: This is an initial summary. Please check the facts about the repositories, I am not 100% familiar with them. Maybe a note should be made of Steam's odd presence and the reason for "apt-get & sudo apt-get"? Something like "many standard popular repository programs (and Steam, a game platform, which usually is not used for most programs)"? I'm not sure how to word it. There need to be wiki links, and the list currently in the #Trivia section should be incorporated in the main explanation (with wiki links). Is there a joke on the name of the program which could be read like Installish and what is a ".sh" file. How would you execute this script on a program - like it is mentioned in the explain would result in that the programs name is replace with the "$1" when the script runs? If you can address this issue, please edit the page! Thanks. |
Most users of computers today are used to simple, easy installation of programs. You just download a .exe or a .dmg, double click it, and do what it says. Sometimes you don't even have to install anything at all, and it runs withou any installation.
However, when things are more "homebrew", for example downloading source code, things are more complicated. You have to work with "build environments" and "makefiles", and command line interfaces. To make this process simpler, there exist repositories of programs which host packages of source code and the things needed to build it. When you download the package, it automatically does most of the work of building the code into something executable. However, there are many such repositories, such as "pip" and "brew", among others listed in the comic. If you only know the name of a program or package, you may not know in which repository(ies) it resides.
The script provided in the comic attempts to fix this problem, by giving a "universal install script". It accepts the name of a program when you run it as an argument. This value is then referenced as "$1" (argument number 1) for a large number of commands. Everywhere the script says "$1", it substitutes in the name of the package you gave it. The end result is the name being tried against a large number of software repositories and package managers, and hopefully, at least one of them will be appropriate and the program will be successfully installed. Near the end, it even tries changing the current working directory to that which is assumed to hold the package to be installed, and then runs several commands which build the program from source code.
All in all, this script would probably work; it runs many standard popular repository programs and package managers, and runs the nearly-universal commands needed to build a program.
One of the more subtle jokes in the comic is the inclusion of apt-get
and sudo apt-get
in the same script. In most cases this would be redundant as the sudo
command is just to add admin permissions. This could be a reference to a joke in the Linux community about forgetting to include the sudo command. An example of this joke being used elsewhere was a viral tweet that showed a workaround for the issue. Sudo has also been used both by Randall in 149: Sandwich and by Bill Amend to force Randall to let him appear on xkcd with 824: Guest Week: Bill Amend (FoxTrot).
Another explanation for this could be that plain "apt-get" is for Debian, while Ubuntu etc. use sudo.
The tool curl
downloads files from the network (e.g., the Internet). Used like curl http://xkcd.com/
it downloads the xkcd main page and displays the HTML source code. The pipe |
in the script attaches the output of the command before the pipe to the input of the command after the pipe. Both commands are executed concurrently. bash
is a popular shell for Unix-like operating systems. The line curl "$1" | bash
tries to download a file from the network and to execute the download directly.
The use of & at the end of each line causes the shell interpreter to execute the commands in parallel (asynchronously) instead of sequentially. Even if single commands fail, the rest of them will be executed. Note this is even the case for the final commands which attempt to change to the installed package, probably the only reason why this may not work completely for packages that do need compiling after being downloaded. (However, just running this script again would probably do the trick.)
There appears to be a bug with the & at the end of the "git clone" line; since a git repository typically contains program source code, not executables, it may have been intended to retrieve the source code with git and then compile and install the program in the next line. In this case, the single & should be replaced with &&, an operator that will run the second command only if the first one has completed successfully. This plays into a second bug on the "configure" line, where the placement of the & means that only the "make install" command will be run asynchronously after the "configure" and "make" steps have finished in sequence. To make success as likely as possible, the two lines should be like this:
git clone https://github.com/"$1"/"$1" && (cd "$1"; ./configure; make; make install) &
The title text mentions the possibility that the same program may be in multiple repositories, so in this case, the script will download and install several versions, or it may fail on a number of repositories, in which case usually nothing bad happens. Since all the commands come from different operating systems, versions, or distributions, it is not very likely that more than one will work (with the exception of pip/easy_install and the two forms of apt-get) or even exist on the same system. It mentions that adding a way of automatically saying "yes" to questions asked during the different repository-fetching programs' running, by making them read input from another program that writes a (nearly) endless stream of "y"s, could simplify things further. 2>/dev/null
redirects the second output stream (the "error stream") to the null device driver, which discards all writes to it.
Transcript
- [In the panel is a shell script which, unusual for xkcd, uses only lower case. At the top the title of the program is inlaid in the frame, which has been broken here.]
- Install.sh
- #!/bin/bash
- pip install "$1" &
- easy_install "$1" &
- brew install "$1" &
- npm install "$1" &
- yum install "$1" & dnf install "$1" &
- docker run "$1" &
- pkg install "$1" &
- apt-get install "$1" &
- sudo apt-get install "$1" &
- steamcmd +app_update "$1" validate &
- git clone https://github.com/"$1"/"$1" &
- cd "$1";./configure;make;make install &
- curl "$1" | bash &
Trivia
- pip and easy install are package managers for Python
- brew is the successor/replacement for MacPorts and a package manager for OS X
- npm is the node package manager that maintains node.js packages
- yum is the package management tool for Red Hat Enterprise Linux and some derivatives.
- dnf is the package management tool for Fedora since version 22.
- pkg is the package management tool on BSD systems
- apt-get is the package management tool of choice on debian and derivatives (eg Ubuntu)
- steamcmd refers to Steam, the computer game client
- git is the revision control software used for eg. the linux kernel and gained a lot of traction through the github plattform
- configure/make/make install refers to the default way of compiling software from source (on Linux/Unix)
- curl is a tool for loading data via http:// (eg from a website), this data is then pushed to the shell interpreter (in order to install).
- Note: While this is a security nightmare, the Nvidia drivers for Linux were (but no longer are) installed like that
![comment.png](/wiki/images/0/03/comment.png)
![comment.png](/wiki/images/0/03/comment.png)
![Icons-mini-action refresh blue.gif](/wiki/images/e/e5/Icons-mini-action_refresh_blue.gif)
Discussion
A few comments:
- curl | sh is still a common way to install things like package managers. Until you have Homebrew, or pip (for older versions of Python that didn't bootstrap it), etc., you can't use a package manager to install it, so they usually give you a one-liner to download and run a shell script that installs the package manager. Of course this isn't an issue for linux distros (which, unlike OS X, come with a built-in package manager).
- Mac users probably only interact with Steam through its GUI, but on linux, running steamcmd is more common. And this command will install a game that's in your library but not downloaded yet.
- I don't know why _only_ apt gets a sudo, but for brew, and for typical installations of Python on a Mac, you don't want or need sudo; they encourage you to leave the relevant directory writable by your normal user account.
- This script only handles the popular package managers on OS X and current popular linux distros. No port for FreeBSD, no Choco for Windows, etc. In fact, if you try it on Windows, you should get an error message telling you that you've ruined the joke by trying to extend it.
--162.158.255.82 10:44, 11 March 2016 (UTC)
Also, docker is a deployment tool for deploying isolated, complete applications. For example, instead of just installing the Python scripts to run your web server behind nginx, you'd deploy nginx, Python, the modules you need for each, the appropriate configurations, a variety of tools the server depends on, and your scripts all as one big hunk of stuff. The docker website probably explains it better. :) --162.158.255.82 10:50, 11 March 2016 (UTC)
- Errors
He forgot the .git on the end of the git clone command.
--173.245.54.53 11:16, 11 March 2016 (UTC)
- Actually, the command works fine anyway. I don't know whether it's git or GitHub which works around this. 141.101.75.161 11:46, 11 March 2016 (UTC)
- Really? I've been typing 4 more characters than I needed to all this time. 173.245.54.10 16:29, 11 March 2016 (UTC)
Also, the TLD in the curl. And, the install script would probably be at /install.sh, and use sh not bash.
Sh is generally preferred in scripting anyway since it comes on all *nix systems by default. Bash is on a very large number of systems, but not all.
Apt-get should have the -y flag.
If installing a program, npm should be given the -g flag to install globally instead of just in this directory.
Most programs print errors (as would arise if a package did not exist) to the console even if they are run with an & to indicate it should not be attached to the session. In this case, it should be &>/dev/null.
The program as a whole ignores previous programs and continues anyway. If it was found in one package manager, it would be a a very bad idea to write over it with another package manager's copy. This is part of the point of the comic, as is noted in the title text, but it's still an error. --173.245.54.53 11:38, 11 March 2016 (UTC)
He forgot cpanm. :) 108.162.217.17 16:02, 11 March 2016 (UTC)
He also left off emerge for Gentoo users. 198.41.235.47 19:08, 11 March 2016 (UTC)
- Question
That whooshing sound you heard was the Linux-y stuff going way over my head, but could part of the joke be that he's trying to install money? With all the $1's in the script? 173.245.54.53 15:47, 11 March 2016 (UTC)
- No, all those $ are just part of the scripting language -- the $1's get replaced with the name of the program you're trying to install. There are so many $ simply because he's included so many install commands, each one of which needs the name of the program.-boB (talk) 16:00, 11 March 2016 (UTC)
- On that note, would any of these fail or would it not just be easier to use `$@`? xerxesbeat (talk) 19:27, 11 March 2016 (UTC)
- Inaccurate Description of &&
The description formerly described the usage of &&:
"This bug could be indicative that Randall wanted to use && throughout the whole script. This would make the installation trying sequentially and the first successful install stops the script and will not install multiple versions of the same software."
This is false. The && operator will *quit* when it encounters the first command that *fails*. The operator that behaves as described is ||. With that said, it is obvious that Randall did not intend this, especially because the title text mentions what happens when multiple versions are installed.
108.162.216.64 16:23, 11 March 2016 (UTC)
Won't work on Arch Mikemk (talk) 00:05, 12 March 2016 (UTC)
- i am very disappointed that that does not read "doesn't work..." --162.158.153.29 12:58, 14 March 2016 (UTC)
- Added clarification on bash scripts
I've added a few lines addressing the concerns formerly displayed in the incomplete tag. Hopefully my edits will be easier for the layman to understand. Please let me know if this needs further attention. AfroThundr3007730 (talk) 12:30, 16 March 2016 (UTC)
He totally forgot to include "urpmi $1" --108.162.216.11 03:33, 17 March 2016 (UTC)
- asdf
I was amazed to notice someone actually created such a program, esoterically called asdf, which supports multiple languages like that. --Anarcat (talk) 17:39, 22 October 2018 (UTC)
- Meta Package Manager
There is a CLI called Meta Package Manager that is trying to solve this issue. Source: comment on Hacker News.