How to speed up Rubocop's startup time

If you have a fairly complex .rubocop.yml file, you may find that later versions take forever to start up or report results, and it seems like it hanging when you run it manually from the CLI.

The cause for this may be due to you having custom exclusion rules. When you exclude files from Rubocop ala:

AllCops:
  Exclude:
    - 'db/fixtures/**/*'
    - 'db/migrate/*.rb'
    - 'db/schema.rb'
    - 'db/seeds.rb'
    - 'Gemfile.lock'
    - 'bin/*'
    # ... etc... etc... 

Rubocop will then ignore any of its own exclusion rules, and land up scanning node_modules, cache dirs, log dirs, and even your .git folder!

You can fix this by adding the following into your .rubocop.yml file.

inherit_mode:
  merge:
    - Exclude

Rubocop will now merge in its default exclusion list alongside your own.

Why this is not the default behaviour eludes me. :|

Debugging PDFKit and WKHTMLToPDF

When PDFKit errors out with an unhelpful error like: RuntimeError (command failed (exitstatus=1): /bin/wkhtmltopdf --page-size A4 - -):, try the following:

• Add config.verbose = true to its configuration
• Add load_error_handling: 'ignore' to its config.default_options
• Save the rendered HTML to a file by modifying pdfkit.rb’s to_pdf method with this just before the invoke declaration.

open('/tmp/pdfkit-source.html', 'a') { |f| f.puts @source.to_s }

Once you’ve got the html saved, you can inspect it for errors, and manually run wkhtmltopdf on the CLI using it as input like so:

cat /tmp/pdfkit-source.html | <the wkhtmltopdfbin command that is failing> out.pdf

Hopefully one of these techniques can highlight where your problems lie, and get you to a resolution.

Quickly lock MacOS with the TouchID button

If you press the TouchID button with your main finger, it does nothing, but did you know that if you press the TouchID button with a finger that MacOS does not have a fingerprint record for, your screen will immediately lock?

This is much quick than using hot corner — which, let’s be honest — gets triggered accidentally more often than not.

It’s also one less shortcut key-combo you’ll need to remember going forward.

This works with both laptop and external keyboard TouchID buttons.

A simple way to keep the latest n docker images

We recently had a CI server run out of space because it had too many images on it that were no longer relevant.

This handy command can be used to delete all but the newest number of images. Slap it into cron and never worry about your disk filling up from too many images

docker images --format "{{.Repository}}:{{.Tag}}" | grep my-fancy-image | tail --lines +21 | xargs --no-run-if-empty docker image rm

The example above will remove all but the latest 20 versions of my-fancy-image.

Take note to add 1 to the amount you want to keep when passing that value to tail, e.g. if you want to keep the latest 5, pass +6 to tail --lines.

Configure Tmux to use 1 as its first window index

Tmux will start its first window at index 0, which can be unintiutive when you visually associate your windows with the number row on your keyboard. i.e. your first Tmux window is the last number on the number row (0), and the second window is the first number on the row (1).

This config in ~/.tmux.conf will make 1 the starting index so that Tmux windows and the number row both grow from left to right in an intuitive manner.

# Start windows and panes at 1, not 0
set  -g base-index      1
setw -g pane-base-index 1

How to correctly specify files when building gems

I recently found one of my docker images had 19MB of test files in the Ruby gems directory. When building a docker image, you want it as small as possible.

I wondered why some gems in there had tests included, and others did not? Here’s what I discovered…

When building a gem, it looks in your gemname.gemspec file for a list of files to include.

The default is a command that includes every file. It’s better to be specific about what files you include, instead of the default shotgun approach

In your gemspec file:

BAD

s.files = `git ls-files -z`.split("\0")

GOOD

s.files = Dir.glob("{lib}/**/*") + %w(LICENSE README.md CHANGELOG.md)

How to use TouchID for sudo access

If you’re on a Mac with TouchID, you can easily configure it to prompt for a TouchID scan alongside normal password auth when running a command via sudo.

• Edit /etc/pam.d/sudo with sudo
• Add this as the first line underneath the comment at the top:

auth sufficient pam_tid.so

• Save the file, and the next time you require sudo, you should be greeted with a TouchID prompt.

Notes

• I haven’t tested this on the new magic keyboards with integrated TouchID, but there’s no reason to believe it won’t work with these keyboards too.
• You may need to reapply this config when MacOS updates

How to check with Ansible if rails must migrate

Running a migration can be a lengthy process which may require app restarts. It’s best to avoid blindly running one just in case and only running them when required.

Using Ansible, we can achieve this with these two tasks:

- name: Check for outstanding migrations
  ansible.builtin.shell:
    cmd:   bundle exec rails db:migrate:status | grep --silent --perl-regex "^\s+down\s+\d+"
    chdir: /to/my/app/dir
  register:     migration_status
  changed_when: migration_status.rc == 0
  failed_when:  migration_status.rc < 1
  run_once:     true

- name: Run any outstanding migrations
  ansible.builtin.shell:
    cmd:   bundle exec rails db:migrate
    chdir: /to/my/app/dir
  when:    migration_status.rc == 0
  notify:
    - Some handler that restarts things

The first task runs rails db:migrate:status and searches for any down, i.e. outstanding migrations. It registers the return, and does three key things:

1. Considers the task as changed if the exit status is 0
2. Considers the task as failed if exist status is > 1, as 1 means no match, meaning no migrations and therefore not a failure condition.
3. Only runs the task once, so if you have a large inventory, you don’t waste time.

The 2nd task only runs if the first’s rc is 0, and then notifies a handler to restart the app if needed.

Refactor multiple SASS properties at once

Whilst working on a project, we noticed a SASS upgrade started throwing a lot of deprecation notices:

DEPRECATION WARNING on line 3 of /app/assets/stylesheets/css_table.css.sass:
Old-style properties like ":float left" are deprecated and will be an error in future versions of Sass.
Use "float: left" instead.

Simple enough if there are one or two, but updating thousands of properties doesn’t sound like fun.

Using VSCode, open a file or project based Find & Replace. Make sure you have selected the .* button.

Add this to the find dialog:

^(\s+)(:)(\w+-?\w+)

Add this to the replace dialog:

$1$3:

And hit the replace or replace-all button. Done!

The regex does the following:

• Find lines starting with some whitespace, and match it.
• Identify any :’s and match them too (probably not needed).
• Find any words that may or may not be hyphenated, and match them as well.

The replace does the following:

• First replace with the matched whitespace ($1)
• Then replace with the matched word ($2)
• Then add a : after the word.

Reference keyword args inside method signatures

Ruby’s keyword arguments are pretty flexible to begin with, but I recently discovered you can reference previously set keyword arguments inside your method definition.

Here’s an example:

def query(type:, response_field: "#{type}_response")
  response = run_query_against(type)
  extract_payload(response, response_field)
end

query(type: "accounts") # Will set response_field to "accounts_response"
query(type: "foo", response_field: "bar") # Will use foo as type, and bar as response_field

From my experiments, it seems that each keyword argument has the scope of the previously defined keyword arguments available to it. This means that you cannot reference a keyword defined later in the signature, e.g.

def query(response_field: "#{type}_response", type: "accounts")
 # This raises a NameError: undefined local variable or method `type'
end

The above will raise an exception, even if there is a default value supplied in the signature.

Save 100s of MBs with Docker multi-stage builds

If you’re building a Docker image to serve a frontend that is compiled down to static files, you can save hundreds of MBs - and avoid some serious security issues - by excluding node_modules and everything else required to compile your Nodejs app by using multi-stage builds.

FROM node:15-something AS builder # use AS to define a name for this build stage

COPY    package*.json /app/
WORKDIR /app
RUN     npm install

COPY . /app/
RUN  npm run build # or whatever your build command is

# Now we start a new build stage
FROM nginx:1-something AS runtime

COPY --from=builder /app/dist/index.html /usr/share/nginx/html/ # Pay attention to the --from command which
COPY --from=builder /app/dist/static     /usr/share/nginx/html/ # which references the previous stage's name

WORKDIR /usr/share/nginx/html

Here’s a before and after:

REPOSITORY   TAG     IMAGE ID     CREATED          SIZE
my_node_app  latest  2a0e6ea0c2fb 51 minutes ago   168MB
my_node_app  fatty   f262d57f5c62 4 hours ago      1.1 GB

The new image only has NGINX, and the static files it needs, and none of the Nodejs bloat required for compiling the app at all, and coming in at only 15% of its original size!

Find the result of an evaluation in an Enumberable

Ruby allows mapping through an array to return a transformed array based on an evaluation in the block. It also allows finding a value in an array.

Ruby unfortunately does not have a single method that allows you to do both, i.e. loop through a list and return the first transformed value that meets a truthy condition.

To achieve this, we can leverage Ruby’s lazy enumerability.

my_val = 5
arr    = [->(val) { val * 3 }, ->(val) { val * 2 }, ->(val) { val * 7 }]

arr.lazy
   .map  { |my_proc|  my_proc.call(my_val) }
   .find { |proc_val| proc_val % 10 == 0 }
 # => 10 (the returned value of the second element
 # in the list. Element 3 is not processed.

Many thanks to Dylan Bridgman (@dylanbr) from the #ruby channel on the ZATech Slack for sharing this neat method.

Browse papertrail changes without object_changes

If you are using PaperTrail to track changes to your data, but are not storing individual changesets, you can still see what changed from version to version using the snippet below:

def hash_diff(from, to)
  from
  .dup
  .delete_if { |k, v| to[k] == v }
  .merge!(to.dup.delete_if { |k, v| from.key?(k) })
end

def present_diff(from_diff, to_diff)
  [].tap do |arr|
    arr << "FROM: #{from_diff}"   if from_diff.present?
    arr << "TO:   #{to_diff}\n\n" if to_diff.present?
  end
end

def list_papertrail_changes_for(obj)
  obj.versions.each_with_index.flat_map do |ver, idx|
    next if ver.object.nil?

    this_ver     = YAML.load(ver.object)
    next_version = obj.versions[idx + 1] || ver
    next_ver     = YAML.load(next_version.object)

    from_diff = hash_diff(this_ver, next_ver)
    to_diff   = hash_diff(next_ver, this_ver)

    present_diff(from_diff, to_diff)
  end.compact
end

puts list_papertrail_changes_for(thing)

FROM: {"ip_address"=>"", "updated_at"=>"2021-04-06T07:32:26.432Z"}
TO:   {"ip_address"=>"10.27.57.194", "updated_at"=>"2021-04-06T08:14:03.506Z"}

FROM: {"ip_address"=>"10.27.57.194", "updated_at"=>"2021-04-06T08:14:03.506Z"}
TO:   {"ip_address"=>"", "updated_at"=>"2021-04-09T13:47:07.485Z"}

Environment variables in crontabs

If you need to set an env var that is used for a bunch of cron jobs, you can set it once at the top of your crontab, and cron will be smart enough to make it available to all jobs specified.

PATH=/my/extra/paths/bin:$PATH

# MIN HOUR DOM MONTH DOW CMD Do something in a strange path
0     12   *   *     *   run_this_command_that_lives_somewhere_strange -rf *

Quickly find your largest docker images

Here’s a handy command to stuff into a shell alias somewhere:

docker images --format '{{.Size}} {{.Repository}} {{.Tag}} {{.ID}}' | sort --human-numeric-sort | column -t

The --format param on the docker command takes a Go template string that will print the size, name, tag and id of your images.

It then gets piped to the sort command with a switch for that understands the difference between MB and GB, and is finally then piped to a handy utility that attempts to format input as a table. Example output:

104MB redis                     latest            ef47f3b6dc11
109MB nginx                     1.15              881bd08c0b08
110MB hashicorp/terraform       0.14.4            18e46faca2ff
113MB certbot/certbot           latest            4bdc1514009b
118MB k8s.gcr.io/kube-proxy     v1.19.3           cdef7632a242
119MB k8s.gcr.io/kube-apiserver v1.19.3           a301be0cd44b
132MB nginx                     1.19.1            0901fa9da894
133MB nginx                     1.19.5            bc9a0695f571
133MB nginx                     latest            ae2feff98a0c
147MB ruby                      2.6.6-slim-buster ab543f69a598
183MB phusion/baseimage         0.11              14e6c478b00a
205MB ruby                      2.3-slim-stretch  6c82e230dca6
222MB kubernetesui/dashboard    v2.0.0            8b32422733b3
222MB ruby                      2.3-slim-jessie   4804bf5dd1e3
240MB osixia/phpldapadmin       latest            afce031a5c02
253MB k8s.gcr.io/etcd           3.4.13-0          0369cf4303ff
265MB postgres                  9.6.1             4023a747a01a
313MB postgres                  12                b97bae343e06
314MB postgres                  latest            f51c55ac75ed
372MB mysql                     5.7               ee7cbd482336
544MB mysql                     latest            0d64f46acfd1
692MB jboss/keycloak            12.0.3            eb71e4d34b56
858MB jboss/keycloak            4.5.0.Final       7874d3a082ee

Easily reuse commands in your shell

Two techniques I love using when working with commands in the shell are quickly running a previous command in my history, and rerunning a previous command, but with sudo.

To quickly rerun a command in your history, simply run !<history_number>, e.g.

$ history

 1451  git pull
 1452  git diff docker-compose.yml
 1453  git checkout -- docker-compose.yml
 1454  vim docker-compose.yml
 1455  docker-compose down
 
$ !1454 # reruns vim docker-compose.yml

Next, I often run a command as a user, when I meant to run it as root, !! is a shell expression which expands to the last command. e.g.

$ service docker restart

Failed to restart docker.service: Insufficient privileges

$ sudo !! # will expand to sudo service docker restart

Toggle and focus VSCode's integrated terminal

Add these shortcut bindings to toggle and focus the terminal. Hitting ⌘+t again will hide the terminal.

{
  "key":     "cmd+t",
  "command": "workbench.action.terminal.toggleTerminal"
}

and

{
  "key":     "cmd+t",
  "command": "workbench.action.terminal.focus",
  "when":    "!terminalFocus"
}