Today I Learned

A Zero One initiative

How to speed up Rubocop's startup time

If you have a fairly complex .rubocop.yml file, you may find that later versions take forever to start up or report results, and it seems like it hanging when you run it manually from the CLI.

The cause for this may be due to you having custom exclusion rules. When you exclude files from Rubocop ala:

AllCops:
  Exclude:
    - 'db/fixtures/**/*'
    - 'db/migrate/*.rb'
    - 'db/schema.rb'
    - 'db/seeds.rb'
    - 'Gemfile.lock'
    - 'bin/*'
    # ... etc... etc...

Rubocop will then ignore any of its own exclusion rules, and land up scanning node_modules, cache dirs, log dirs, and even your .git folder!

You can fix this by adding the following into your .rubocop.yml file.

inherit_mode:
  merge:
    - Exclude

Rubocop will now merge in its default exclusion list alongside your own.

Why this is not the default behaviour eludes me. :|

Debugging PDFKit and WKHTMLToPDF

When PDFKit errors out with an unhelpful error like: RuntimeError (command failed (exitstatus=1): /bin/wkhtmltopdf --page-size A4 - -):, try the following:

  • Add config.verbose = true to its configuration
  • Add load_error_handling: 'ignore' to its config.default_options
  • Save the rendered HTML to a file by modifying pdfkit.rb’s to_pdf method with this just before the invoke declaration. 
open('/tmp/pdfkit-source.html', 'a') { |f| f.puts @source.to_s }

Once you’ve got the html saved, you can inspect it for errors, and manually run wkhtmltopdf on the CLI using it as input like so:

cat /tmp/pdfkit-source.html | <the wkhtmltopdfbin command that is failing> out.pdf

Hopefully one of these techniques can highlight where your problems lie, and get you to a resolution.

Beware of Stale Closure Data in React

Something we continually take for granted is that setState is actually an asynchronous method, and closures in JavaScript can get end up getting a stale version of state (if they’re using any) because of this.

This is particularly painful when your next version of some state depends on the previous version (Eg. A counter + 1).

To ensure that closures don’t experience issues like this, you can provide a callback to setState instead of a fixed value.

Eg. Instead of:

[count, setCount] = useState(0);
const increment = () => setCount(count + 1);

Do this:

[count, setCount] = useState(0);
const increment = () => setCount((prevCount) => prevCount + 1);

Your increment closure is now a little more resilient to asynchronous events, and since it doesn’t refer to some stale version of state, it will update the next version accordingly.

Add comments to package.json

Progressively upgrading dependencies in the JS world is a little different compared to Ruby.

When updating a Ruby gem inside a gemfile, you might want to leave notes. For example:

gem 'some-gem', '~> 1.2.3' # TODO: Update once we're on Ruby 3.1

The comment is useful since it shares knowledge related to upgrading this dependency.

Javascript, or JSON, works a little differently. No comments are allowed inside the package.json file. Instead to achieve roughly the same result you can do this:

...
"//": [
    "TODO: Upgrade stylelint-config-sass-guidelines once we're on Node 11+",
    "TODO: Upgrade mini-css-extract-plugin once we're on 12.13+ and webpack 5+",
    "TODO: Upgrade style-loader once we're on 12.13+ and webpack 5+"
  ],
"dependencies": {
  ....
}

"//" will never be used by npm for any purpose, and is reserved for comments.

Quickly lock MacOS with the TouchID button

If you press the TouchID button with your main finger, it does nothing, but did you know that if you press the TouchID button with a finger that MacOS does not have a fingerprint record for, your screen will immediately lock?

This is much quick than using hot corner — which, let’s be honest — gets triggered accidentally more often than not.

It’s also one less shortcut key-combo you’ll need to remember going forward.

This works with both laptop and external keyboard TouchID buttons.

A simple way to keep the latest n docker images

We recently had a CI server run out of space because it had too many images on it that were no longer relevant.

This handy command can be used to delete all but the newest number of images. Slap it into cron and never worry about your disk filling up from too many images

docker images --format "{{.Repository}}:{{.Tag}}" | grep my-fancy-image | tail --lines +21 | xargs --no-run-if-empty docker image rm

The example above will remove all but the latest 20 versions of my-fancy-image.

Take note to add 1 to the amount you want to keep when passing that value to tail, e.g. if you want to keep the latest 5, pass +6 to tail --lines.

Finding missing/associated records in Rails

Rails has some built in methods to help you find records which may have missing associated records.

For example, let’s assume we have an app where an Invoice has many Payments, and we wanted to find all invoices which have no payments:

Invoice.where.missing(:payments)

We’ll get back a collection of all invoices which have no payments.

Much cleaner than writing messy joins!

Also, on the flip side you can find all invoices with payments like this:

Invoice.where.associated(:payments)

Extending Rake Tasks

You can extend rake tasks to add extra functionality.

For example, let’s extend the rails db:migrate task to automatically include schema migrations and data migrations.

# Inside lib/tasks/db.rake
# Remove the old migrate task
Rake::Task['db:migrate'].clear

namespace 'db' do
  # Redefine the migrate task to invoke with data migrations
  task migrate: ['migrate:with_data']
end

Now whenever you execute rails db:migrate, the task will include schema + data migrations.

Additionally, you may want to invoke some extra task after another has been executed. To do this, you can enhance a task like this:

# Inside lib/tasks/db.rake
Rake::Task['db:migrate'].enhance do
  Rake::Task['db:do_some_custom_thing'].invoke
end

Now your custom task will be invoked after rails db:migrate is invoked.

Simpler conditional HTML classes with class_names

Coming from JS and React you may be familiar with a package called classnames. It takes an object where the keys represent CSS classes, and the values are booleans which determine if the key should be loaded or not.

Eg. classnames(‘foo’, { bar: true, baz: false })

Returns: ‘foo bar’

Rails includes its own helper which does this same thing. You can call class_names or token_list:

Eg. class_names(‘foo’, { bar: true, baz: false })

Returns: ‘foo bar’

Configure Tmux to use 1 as its first window index

Tmux will start its first window at index 0, which can be unintiutive when you visually associate your windows with the number row on your keyboard. i.e. your first Tmux window is the last number on the number row (0), and the second window is the first number on the row (1).

This config in ~/.tmux.conf will make 1 the starting index so that Tmux windows and the number row both grow from left to right in an intuitive manner.

# Start windows and panes at 1, not 0
set  -g base-index      1
setw -g pane-base-index 1

Stimulus Use

I was getting rather annoyed when using modals with stimulus, especially when it came to actually closing them. I didnt like the way I wrote the general function to close the modals so I decided to refactor it. In doing so I came across a package that would solve it all. The package is called stimulus-use (https://github.com/stimulus-use/stimulus-use). It enables new lifecycle behaviors to your stimulus controllers. Now I can close my modals like so, instead of trying to check if my modal has a click event outside of it:

import { Controller } from 'stimulus'
import { useClickOutside } from 'stimulus-use'

export default class extends Controller {

  connect() {
    useClickOutside(this)
  }

  clickOutside(event) {
    event.preventDefault()
    this.modal.close()
  }
}

Default Controller Params

On occasion you may need to assign default values to the params required within a controller. For example, assume we’re working with todos and we want to assign a todo to the current user (by default, unless another user is specified), then I’ve usually seen some people handle default param values like this:

def todo_params
  params
    .require(:todo)
    .permit(:subject, :description, :user_id)
    .tap { |pms|  pms.user_id ||= current_user.id }
end

However, TIL that you can actually make use of a method called with_defaults:

def todo_params
  params
    .require(:todo)
    .permit(:subject, :description, :user_id)
    .with_defaults(user_id: current_user.id)
end

Much cleaner!

Also, with_defaults is defined for ActionController::Parameters and Hash. I find with_defaults much cleaner, and more intention revealing, than reverse_merge!

Fun with Recursive Anonymous Functions

Did you know that an anonymous function can call itself?

All you need is something like this:

factorial_of = -> (n) { n <= 1 ? 1 : n * factorial_of.call(n - 1) }
factorial_of.call(5)
# => 120

This came in handy when I had a recursive method which needed to be wrapped inside an enumerator to navigate a very complex hierarchy of objects.

Instead of something as complex as that, let’s wrap our anonymous factorial function around an enumerator:

factorial_of = -> (n) { n <= 1 ? 1 : n * factorial_of.call(n - 1) }

f = Enumerator.new do |y|
  num = 0
  loop do
    y << factorial_of.call(num)
    num += 1
  end
end

f.next
=> 1
f.next
=> 1
f.next
=> 2
f.next
=> 6

Now we can fetch the next number whenever we want, or enumerate over the numbers until we don’t have any left (which is infinite in this case)

Slide an Organiser into another Organiser's DM's

When calling an Organiser within another Organiser with light-service, a really neat way to do it is to splat the actions from the desired organiser into the other like this:

module Users
  class UpdateProfile
    extend ::LightService::Organizer

    def self.call()
      with(
        # some params go here
      ).reduce(actions)
    end

    def self.actions
      [
        # some actions go here
        *Users::UpdatePreferencesAndNotify.actions,
        # some more actions go here
      ]
    end
  end
end

I needed to call an organiser responsible for updating a completeness score on the user and also running the organiser responsible for updating the users attributes and awarding a badge based on the score. This allowed me to still segment the logic while not cluttering the organiser with more actions.

See https://github.com/adomokos/light-service/issues/160 for reference. Also it’s not necessary to use the extend LightService::Action in the organisers

Prevent commits to main/master with git hooks!

You can use git hooks to keep you safe when you aren’t thinking properly and decide to commit something to main/master.

It would be nice if we could create a hook which would prevent commits to main/master unless we give it some override. Like a crimesjohnson prefix.

As a start I decided to make a global hook for all of my projects with this command:

git config --global core.hooksPath ~/githooks

I then created a file (within ~/githooks) for a special kind of git hook and named it commit-msg.

#!/bin/sh

commit_msg=$(cat "${1:?Missing commit message file}")
branch="$(git rev-parse --abbrev-ref HEAD)"

if ([ "$branch" = "main" ] || [ "$branch" = "master" ]) && [[ "$commit_msg" != *"crimesjohnson:"* ]]; then
  echo "You can't commit directly to main/master branch"
  exit 1
fi

Don’t forget to make it executable: chmod +x ~/githooks/commit-msg

When you create a commit message, it seems like the message is saved to a temp file while the commit-msg hook runs. We get the commit message by cating it.

We also use a git command to get the name of our branch.

Using these 2 bits of info, we can then write a simple if statement to keep us safe!

How to correctly specify files when building gems

I recently found one of my docker images had 19MB of test files in the Ruby gems directory. When building a docker image, you want it as small as possible.

I wondered why some gems in there had tests included, and others did not? Here’s what I discovered…

When building a gem, it looks in your gemname.gemspec file for a list of files to include.

The default is a command that includes every file. It’s better to be specific about what files you include, instead of the default shotgun approach

In your gemspec file:

BAD

s.files = `git ls-files -z`.split("\0")

GOOD

s.files = Dir.glob("{lib}/**/*") + %w(LICENSE README.md CHANGELOG.md)

Find your notes within your Rails app

Leaving TODOS and other notes sometimes feels like throwing trash onto a pile which you may never come back to.

You can find all notes you’ve left inside a Rails using rails notes.

This will search app, config, db, lib, and test directories for FIXME, OPTIMIZE, and TODO annotations.

Running rails notes will return something like:

app/actions/auth/creates_new_user_from_open_id_payload.rb:
  * [22] [TODO] Find a way to create users linked to accounts?
  * [27] [TODO] Not using these right now, and they should move to the tokens model

You can see that we receive a list of notes including the file, line number and the note.

You can also filter for specific notes with --annotations. For example let’s filter FIXMEs:

rails notes --annotations FIXME

You can also add custom notes in the Rails config like this:

config.annotations.register_tags("DEPRECATEME", "TESTME")

Adding your own custom notes could be handy when you want to add a note for something like gem/app upgrades in code which might not be tested particularly well, or just difficult to test in general.

How to use TouchID for sudo access

If you’re on a Mac with TouchID, you can easily configure it to prompt for a TouchID scan alongside normal password auth when running a command via sudo.

  • Edit /etc/pam.d/sudo with sudo
  • Add this as the first line underneath the comment at the top: 
auth sufficient pam_tid.so
  • Save the file, and the next time you require sudo, you should be greeted with a TouchID prompt.

Notes

  • I haven’t tested this on the new magic keyboards with integrated TouchID, but there’s no reason to believe it won’t work with these keyboards too.
  • You may need to reapply this config when MacOS updates

Combine Pry and RSpec

RSpec provides a way for you to register aliases for the methods you use to create your specs. You can alias these methods and tie them into predefined hooks using metadata.

For example, let’s say we wanted to have run pry whenever a spec includes some metadata like: pry: true. Instead of adding pry: true after each spec, we can add an alias by doing the following:

RSpec.configure do |config|
  config.alias_example_group_to :pdescribe, pry: true
  config.alias_example_to       :pit,       pry: true

  config.after(:example, pry: true) do |e|
    require 'pry'
    binding.pry
  end
end

We’ve registered 2 aliases. Whenever we prepend p onto a method’s name, we’ll now get the same functionality as adding pry: true to our spec’s metadata.

We’ve tied this alias into an after hook. This hook only runs when pry is set to true in the spec’s metadata. RSpec after hooks always run, even if the spec fails!

Simply prepend p to a spec and pry will run once the spec is finished!

We’ve now created a powerful pry alias to debug our specs in RSpec :)

Ensure clauses

Today I learned about the ensure clause when raising exceptions. I was having trouble cleaning up certain operations, now within the ensure block the code will always get executed whether an exception is raised or not.

def foo
  begin
     raise Exception, "Oh no"
   ensure
     return "Clean me up Scotty"
  end
end

DRY up kwargs using `with_options`

Rails has a really awesome helper to DRY up code which makes use of much of the same kwargs. It’s called with_options and is used like this:

class Account < ActiveRecord::Base
  has_many :customers, dependent: :destroy
  has_many :products,  dependent: :destroy
end

# Becomes this with `with_option`

class Account < ActiveRecord::Base
  with_options dependent: :destroy do
    has_many :customers
    has_many :products
  end
end

Your code is now a lil more DRY and its easier to add, or remove, pesky kwargs as needed.

Another quick example:

class AuthenticatedController < ApplicationController
  before_action :logged_in?,         only: [:show, :edit]
  before_action :create_paper_trail, only: [:show, :edit]

  # ...
end

# Becomes this with `with_option`

class AuthenticatedController < ApplicationController
  with_options only: [:show, :edit] do
    before_action :logged_in?
    before_action :create_paper_trail
  end

  # ...
end

Strict loading with Rails

Rails implements a feature called strict loading. This feature can help catch potential N+1 queries.

Did you know there are a couple of ways to enable strict loading?

1. Per Record

Rails let’s you enable strict loading on specific records at runtime:

post = Post.first

post.strict_loading! # Enable strict loading
post.strict_loading? # Check if strict loading is enabled on this object
# => true

2. Per model

Rails let’s you enable strict loading on all models of a specific type:

class Post < ApplicationRecord
  self.strict_loading_by_default = true

  has_many :comments, dependent: :destroy
end

post = Post.first
post.strict_loading?
# => true

3. Enable per association basis

Rails let’s you define strict loading for specific (or pesky) relations on a model:

class Post < ApplicationRecord
  has_many :comments, dependent: :destroy, strict_loading: true
  has_many :likes,    dependent: :destroy, strict_loading: false
end

4. Application wide

Rails also let’s you enable strict loading all across the board when you’re brave enough for the nuclear option:

# config/environments/development.rb

Rails.application.configure do
# ...
  config.active_record.strict_loading_by_default = true
# ...
end

Now that you know your options, you can enable it based on your appetite/distaste for N+1 queries!

Ruby prepend wrappers

Imagine we have some sort of object (which we don’t own) and we’d like to audit it. You could write an explicit wrapper around the object to audit it, but then you’ve got to keep in mind that you have to use this wrapper whenever you use the service.

You can add new functionality, whilst preserving the old interface and function like this:

class Foo # class we do not own
  def bar(arg)
    sleep arg
    "Running bar with #{arg}!"
  end
end

module FooWrapper
  def bar(arg)
    start_time = Time.now
    result = super
    time_taken = Time.now - start_time

    puts "bar() took more than #{time_taken} seconds" if 2 < time_taken
    result
  rescue
    "bar() failed"
  end
end

Foo.class_eval do
  prepend FooWrapper
end

Foo.new.bar 2.5
# bar() took more than 2.501257 seconds
# => "Running bar with 2.5!"

We’ve created a wrapper, opened up the class and prepended the wrapper. The wrapper adds new functionality, whilst preserving the old.

The downside to this approach is that whenever you call Foo#bar, you will get your added functionality each time. This may, or may not, be an issue depending your circumstance.

Make fancy enums with `enum_for`

Did you know you can make your own enumerators in Ruby?

e = Enumerator.new do |y|
 [1,2,3].each { |i| y << i }
end

e.next    # => 1
e.next    # => 2
e.rewind
e.next => # => 1

Methods like each help convert things to an enumerable (using to_enum), but there is another way to create enums in Ruby. enum_for takes one argument which points to a method to bind to for enumeration. It allows make something enumerable when it technically isn’t, or when we can’t/don’t want to.

class DoReMi
  def sing
    lyrics.each { |l| yield l }
  end

  private

  def lyrics
    [
      "Doe, a deer, a female deer",
      ...
    ]
  end
end

Use enum_for to enumerate at your own pace. Let’s learn to shout the song backwards!

d = DoReMi.new
d.sing { |l| puts l }
# => Doe, a deer, a female deer
# => ...
e = d.enum_for :sing
e.map { |l| l.upcase }.reverse.each { |l| puts l }
# => ME, A NAME I CALL MYSELF
# => ...

# Wait, what's the last lyric again?
e.rewind
e.next.upcase # => "DOE, A DEER, A FEMALE DEER"

`[]` is smarter thank you think!

You’re probably very familiar with using [] on objects which are enumerable, but [] has a few more tricks up it’s sleeve when it comes to strings which you may not know about:

string = "Ruby is a cool language"

Did you know that you can use [] to fetch a substring from a string, and return nil if it doesn’t exist?

string["cool lang"] # => "cool lang"
string["foo"] # => nil

You can even throw a Regex in that bad boy and you’re off to the Regex races!

And for the final trick we’ll make a pesky substring DISAPPEAR!

string["cool"] = "great" # => "great"
string # => "Ruby is a great language"

Next time you’re munging with strings, remember that [] is smarter than you think!

How to check with Ansible if rails must migrate

Running a migration can be a lengthy process which may require app restarts. It’s best to avoid blindly running one just in case and only running them when required.

Using Ansible, we can achieve this with these two tasks:

- name: Check for outstanding migrations
  ansible.builtin.shell:
    cmd:   bundle exec rails db:migrate:status | grep --silent --perl-regex "^\s+down\s+\d+"
    chdir: /to/my/app/dir
  register:     migration_status
  changed_when: migration_status.rc == 0
  failed_when:  migration_status.rc < 1
  run_once:     true

- name: Run any outstanding migrations
  ansible.builtin.shell:
    cmd:   bundle exec rails db:migrate
    chdir: /to/my/app/dir
  when:    migration_status.rc == 0
  notify:
    - Some handler that restarts things

The first task runs rails db:migrate:status and searches for any down, i.e. outstanding migrations. It registers the return, and does three key things:

  1. Considers the task as changed if the exit status is 0
  2. Considers the task as failed if exist status is > 1, as 1 means no match, meaning no migrations and therefore not a failure condition.
  3. Only runs the task once, so if you have a large inventory, you don’t waste time.

The 2nd task only runs if the first’s rc is 0, and then notifies a handler to restart the app if needed.

Refactor multiple SASS properties at once

Whilst working on a project, we noticed a SASS upgrade started throwing a lot of deprecation notices:

DEPRECATION WARNING on line 3 of /app/assets/stylesheets/css_table.css.sass:
Old-style properties like ":float left" are deprecated and will be an error in future versions of Sass.
Use "float: left" instead.

Simple enough if there are one or two, but updating thousands of properties doesn’t sound like fun.

Using VSCode, open a file or project based Find & Replace. Make sure you have selected the .* button.

Add this to the find dialog:

^(\s+)(:)(\w+-?\w+)

Add this to the replace dialog:

$1$3:

And hit the replace or replace-all button. Done!

The regex does the following:

  • Find lines starting with some whitespace, and match it.
  • Identify any :’s and match them too (probably not needed).
  • Find any words that may or may not be hyphenated, and match them as well.

The replace does the following:

  • First replace with the matched whitespace ($1)
  • Then replace with the matched word ($2)
  • Then add a : after the word.

Reference keyword args inside method signatures

Ruby’s keyword arguments are pretty flexible to begin with, but I recently discovered you can reference previously set keyword arguments inside your method definition.

Here’s an example:

def query(type:, response_field: "#{type}_response")
  response = run_query_against(type)
  extract_payload(response, response_field)
end

query(type: "accounts") # Will set response_field to "accounts_response"
query(type: "foo", response_field: "bar") # Will use foo as type, and bar as response_field

From my experiments, it seems that each keyword argument has the scope of the previously defined keyword arguments available to it. This means that you cannot reference a keyword defined later in the signature, e.g.

def query(response_field: "#{type}_response", type: "accounts")
 # This raises a NameError: undefined local variable or method `type'
end

The above will raise an exception, even if there is a default value supplied in the signature.

Save 100s of MBs with Docker multi-stage builds

If you’re building a Docker image to serve a frontend that is compiled down to static files, you can save hundreds of MBs - and avoid some serious security issues - by excluding node_modules and everything else required to compile your Nodejs app by using multi-stage builds.

FROM node:15-something AS builder # use AS to define a name for this build stage

COPY    package*.json /app/
WORKDIR /app
RUN     npm install

COPY . /app/
RUN  npm run build # or whatever your build command is

# Now we start a new build stage
FROM nginx:1-something AS runtime

COPY --from=builder /app/dist/index.html /usr/share/nginx/html/ # Pay attention to the --from command which
COPY --from=builder /app/dist/static     /usr/share/nginx/html/ # which references the previous stage's name

WORKDIR /usr/share/nginx/html

Here’s a before and after:

REPOSITORY   TAG     IMAGE ID     CREATED          SIZE
my_node_app  latest  2a0e6ea0c2fb 51 minutes ago   168MB
my_node_app  fatty   f262d57f5c62 4 hours ago      1.1 GB

The new image only has NGINX, and the static files it needs, and none of the Nodejs bloat required for compiling the app at all, and coming in at only 15% of its original size!

Find the result of an evaluation in an Enumberable

Ruby allows mapping through an array to return a transformed array based on an evaluation in the block. It also allows finding a value in an array.

Ruby unfortunately does not have a single method that allows you to do both, i.e. loop through a list and return the first transformed value that meets a truthy condition.

To achieve this, we can leverage Ruby’s lazy enumerability.

my_val = 5
arr    = [->(val) { val * 3 }, ->(val) { val * 2 }, ->(val) { val * 7 }]

arr.lazy
   .map  { |my_proc|  my_proc.call(my_val) }
   .find { |proc_val| proc_val % 10 == 0 }
 # => 10 (the returned value of the second element
 # in the list. Element 3 is not processed.

Many thanks to Dylan Bridgman (@dylanbr) from the #ruby channel on the ZATech Slack for sharing this neat method.

Query Postgres Array field with SQL & ActiveRecord

If you have, for example a string array column in Postgres you can do a query match on any of the array values columns with a SQL query like this:

SELECT * FROM table WHERE 'value' = ANY(str_field);

This will return records where the string ‘value’ is present in at least one of the array string values stored in str_field

Equivalent query in ActiveRecord

ArModel.where("? = ANY(str_field)", "value")

You can also use ALL to match all values in an array column.

Browse papertrail changes without object_changes

If you are using PaperTrail to track changes to your data, but are not storing individual changesets, you can still see what changed from version to version using the snippet below:

def hash_diff(from, to)
  from
  .dup
  .delete_if { |k, v| to[k] == v }
  .merge!(to.dup.delete_if { |k, v| from.key?(k) })
end

def present_diff(from_diff, to_diff)
  [].tap do |arr|
    arr << "FROM: #{from_diff}"   if from_diff.present?
    arr << "TO:   #{to_diff}\n\n" if to_diff.present?
  end
end

def list_papertrail_changes_for(obj)
  obj.versions.each_with_index.flat_map do |ver, idx|
    next if ver.object.nil?

    this_ver     = YAML.load(ver.object)
    next_version = obj.versions[idx + 1] || ver
    next_ver     = YAML.load(next_version.object)

    from_diff = hash_diff(this_ver, next_ver)
    to_diff   = hash_diff(next_ver, this_ver)

    present_diff(from_diff, to_diff)
  end.compact
end

puts list_papertrail_changes_for(thing)

FROM: {"ip_address"=>"", "updated_at"=>"2021-04-06T07:32:26.432Z"}
TO:   {"ip_address"=>"10.27.57.194", "updated_at"=>"2021-04-06T08:14:03.506Z"}

FROM: {"ip_address"=>"10.27.57.194", "updated_at"=>"2021-04-06T08:14:03.506Z"}
TO:   {"ip_address"=>"", "updated_at"=>"2021-04-09T13:47:07.485Z"}

Environment variables in crontabs

If you need to set an env var that is used for a bunch of cron jobs, you can set it once at the top of your crontab, and cron will be smart enough to make it available to all jobs specified.

PATH=/my/extra/paths/bin:$PATH

# MIN HOUR DOM MONTH DOW CMD Do something in a strange path
0     12   *   *     *   run_this_command_that_lives_somewhere_strange -rf *

Quickly find your largest docker images

Here’s a handy command to stuff into a shell alias somewhere:

docker images --format '{{.Size}} {{.Repository}} {{.Tag}} {{.ID}}' | sort --human-numeric-sort | column -t

The --format param on the docker command takes a Go template string that will print the size, name, tag and id of your images.

It then gets piped to the sort command with a switch for that understands the difference between MB and GB, and is finally then piped to a handy utility that attempts to format input as a table. Example output:

104MB redis                     latest            ef47f3b6dc11
109MB nginx                     1.15              881bd08c0b08
110MB hashicorp/terraform       0.14.4            18e46faca2ff
113MB certbot/certbot           latest            4bdc1514009b
118MB k8s.gcr.io/kube-proxy     v1.19.3           cdef7632a242
119MB k8s.gcr.io/kube-apiserver v1.19.3           a301be0cd44b
132MB nginx                     1.19.1            0901fa9da894
133MB nginx                     1.19.5            bc9a0695f571
133MB nginx                     latest            ae2feff98a0c
147MB ruby                      2.6.6-slim-buster ab543f69a598
183MB phusion/baseimage         0.11              14e6c478b00a
205MB ruby                      2.3-slim-stretch  6c82e230dca6
222MB kubernetesui/dashboard    v2.0.0            8b32422733b3
222MB ruby                      2.3-slim-jessie   4804bf5dd1e3
240MB osixia/phpldapadmin       latest            afce031a5c02
253MB k8s.gcr.io/etcd           3.4.13-0          0369cf4303ff
265MB postgres                  9.6.1             4023a747a01a
313MB postgres                  12                b97bae343e06
314MB postgres                  latest            f51c55ac75ed
372MB mysql                     5.7               ee7cbd482336
544MB mysql                     latest            0d64f46acfd1
692MB jboss/keycloak            12.0.3            eb71e4d34b56
858MB jboss/keycloak            4.5.0.Final       7874d3a082ee

Easily reuse commands in your shell

Two techniques I love using when working with commands in the shell are quickly running a previous command in my history, and rerunning a previous command, but with sudo.

To quickly rerun a command in your history, simply run !<history_number>, e.g.

$ history

 1451  git pull
 1452  git diff docker-compose.yml
 1453  git checkout -- docker-compose.yml
 1454  vim docker-compose.yml
 1455  docker-compose down
 
$ !1454 # reruns vim docker-compose.yml

Next, I often run a command as a user, when I meant to run it as root, !! is a shell expression which expands to the last command. e.g.

$ service docker restart

Failed to restart docker.service: Insufficient privileges

$ sudo !! # will expand to sudo service docker restart

PG Gem fails on Mac OS 11.x (Big Sur)

If after installing pg gem in your ruby project and you get the following error with the pg gem

Library not loaded: libpq.5.dylib

You can fix the linking issues by running the following commands:

cd ~/[your ruby path]/versions/2.7.2/lib/ruby/gems/2.7.0/gems/pg-1.2.3/lib

install_name_tool -change libpq.5.dylib /Library/PostgreSQL/12/lib/libpq.5.12.dylib pg_ext.bundle

You may need to modify the commands above to suite your ruby manager and currently installed versions.