macOS configuration with GNU Programs

I’ve gone over this with colleagues and first-time macOS people enough to warrant a write-up.

Thesis

We all use macOS (Unix; a non-GNU OS) to work (generally) with GNU Linux. macOS programs are slightly different than their GNU counterparts. It makes little sense to write scripts locally on macOS, then expect them to run reliably on Linux. The macOS tools are either:

• not suited to our work, or
• they are shamefully old; example:

$ /bin/bash --version
GNU bash, version 3.2.57(1)-release (x86_64-apple-darwin16)
Copyright (C) 2007 Free Software Foundation, Inc.
$ /usr/bin/git --version
git version 2.14.3 (Apple Git-98) <- not the worst thing but still

Bash 3.x? Bro — the 70’s are over. It’s just not necessary to live in that cave.

This case can be solved by configuring our systems to favor the GNU variants of programs that are the foundation of our work.

The process for system modification follows. It does presume that Apple’s Xcode Command Line Tools is installed. If not, it needs to be. You don’t need the entirety of the Xcode suite, just the command line tools.

Install Support

NOTE: sudo -level system access is required for some configuration steps.

Phase 1: setup your environment properly

$ cat ~/.bash_profile
# REF: https://www.gnu.org/software/bash/manual/bashref.html#Bash-Startup-Files
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

~/.bashrc holds the vast majority of your personalized configurations. I’ll let you do your own reading about the why’s.

Install Homebrew; if you’re not using this package manager yet, start. It makes life wonderfully easy. Install these programs: (in this order)

brew install gnu-sed --with-default-names
brew install coreutils

Configure the macOS (“system”)

Get Todd Thomas’s stories in your inbox

Join Medium for free to get updates from this writer.

Remember me for faster sign in

The system won’t know what to do with any of this stuff unless we change it.

sysPaths='/etc/paths'
sysManPaths='/etc/manpaths'
pathGNU_CORE="$(brew --prefix coreutils)"
# Set path for the GNU Coreutils, et al.
sudo sed -i "\|/usr/local/bin|i $pathGNU_CORE/libexec/gnubin" "$sysPaths"
# Set path for the GNU Coreutils Manuals
sudo sed -i "\|/usr/share/man|i $pathGNU_CORE/libexec/gnuman" "$sysManPaths"
# Move system manpaths down 1 line
sudo sed -i -n '2{h;n;G};p' "$sysManPaths"

These programs will still not be usable until you open a new shell; do that now. Also, the coreutils programs are all prefixed with a ‘g’, like gstat . That’s okay; to TEST:

$ type -P gstat
/usr/local/bin/gstat
$ type -P stat
/usr/local/opt/coreutils/libexec/gnubin/stat
$ ls -l /usr/local/opt/coreutils/libexec/gnubin/stat
lrwxr-xr-x 1 $USER staff 15 Aug  2 20:49 /usr/local/opt/coreutils/libexec/gnubin/stat -> ../../bin/gstat

softlinks have been created in another directory to those prefixed with a ‘g’; fine. Now test sed:

$ type -P sed
/usr/local/bin/sed

Perfect. Let’s see the paths:

$ cat /etc/paths
/usr/local/opt/coreutils/libexec/gnubin
/usr/local/bin
/usr/local/sbin
/usr/bin
/bin
/usr/sbin
/sbin

Because the path for coreutils is at the top and /usr/local/bin is next, the system will favor any tools in these paths over the ones (on the system) below them; the same now applies for man page paths. More installs…

brew install gnu-indent --with-default-names
brew install findutils --with-default-names
brew install gnu-which --with-default-names
brew install wget --with-pcre
brew install gnu-tar --with-default-names
brew install gnu-time --with-default-names
brew install grep --with-default-names
brew install gzip gawk diffutils

Since almost everything homebrew pulls down will be installed to /usr/local/bin there’s not much more path manipulation, but there is some. Test another program:

$ type -P find
/usr/local/bin/find

Great, more installs…

### Install some modern versions of other programs
brew install \    
    git nmap rsync openssl ssh-copy-id watch tree pstree psgrep \
    sipcalc whatmask ipcalc dos2unix testdisk

Build utilities; be careful with this one. cmake relies on Python to get some of its work done. If you have an elaborate Python configuration then make backups.

### Install Build Utilities
brew install make --with-default-names
brew install cmake --with-completion
brew install automake bazel

Install Bash (with helpers):
NOTE: if you’ve previously had bash-completion installed, remove it. That’s for previous (archaic) versions of Bash; removal:

brew uninstall bash-completion

Installation

brew install bash shellcheck dash bash-completion@2

Configuration

# Create a softlink from sh to dash
# For those times you *need* to be POSIX-compliant
ln -sf /usr/local/bin/dash /usr/local/bin/sh

Cause the system to favor the latest GNU Bash/dash

sysShells='/etc/shells'
# System Shells before
grep '^\/' "$sysShells"
# modify the shells file
sudo sed -i "\|^.*bash$|i /usr/local/bin/bash" "$sysShells"
sudo sed -i "\|local|a /usr/local/bin/sh" "$sysShells"
# System Shells after
grep '^\/' "$sysShells"

Switch to your local user account over to GNU Bash

sudo chpass -s '/usr/local/bin/bash' "$USER"
# read the changes back
dscl . -read "$HOME" UserShell

TEST the changes:

$ bash --version
GNU bash, version 4.4.23(1)-release (x86_64-apple-darwin16.7.0)
...
# Check the man page
man bash

The power and the glory 😈

Now, with one additional step, we can also enable bash completions. Edit your ~/.bashrc file and drop this in all the way at the top:

# shellcheck disable=SC2148,SC1090,SC1091,SC2012,SC2139,SC2196
sysBashrc='/etc/bashrc'
bashComps='/usr/local/share/bash-completion/bash_completion'
bashCompsDir='/usr/local/etc/bash_completion.d'
if [[ -f "$sysBashrc" ]]; then
    source "$sysBashrc"
    # enable bash completions
    if [ -f "$bashComps" ]; then
        source "$bashComps"
        if [[ -d "$bashCompsDir" ]]; then
            while read -r compFile; do
                #printf '%s\n' "  \${compFile##*/}"
                source "$compFile"
            done <<< "$(find "$bashCompsDir" -type l)"
        fi
    fi
fi

This behaves as a discrete circuit board. We’re saying:
IF the /etc/bashrc file exists then source it in.
IF the last step was successful, then verify if
/usr/local/share/bash-completion/bash_completion file exists; if so, source that file in. Finally,
IF the /usr/local/etc/bash_completion.d directory exists, then find all bash completion files within and source them in, one at a time.

This will happen quickly for every new terminal window you open.

time source ~/.bashrc
real 0m0.453s
user 0m0.341s
sys  0m0.080s

Now, IF some program you install with brew comes with its own completions file, like kubectl THEN you source-in the ~/.bashrc afterwards and start tabbing.

The whole ~/.bashrc is in a gist if you want to see something abysmal.

However, if you want to see something truly horrific, this is how I can upgrade the macOS in 40 minutes (with data restored) while I’m having lunch. It only took a few hours but, on the upside, it should really be replaced.

I hope this helps. Wanted to get this written up before switching to zsh 😜

Cheers all.