Wiping an AWS Account with aws-nuke

AWS From Scratch with Terraform - Setting up your Root Account for IaC  with Terraform Cloud and Github actions.

Running a kubernetes cluster locally with kubeadm

Running a kubernetes cluster locally with kind

<a href="https://asdf-vm.com/">asdf</a> is a general purpose version manager that
can manage versions of most programming language runtimes through a set
of plugins.
<iframe width="854" height="480" src="https://www.youtube.com/embed/RTaqWRj-6Lg" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture" allowfullscreen></iframe>
With micro-services being all the rage and the ever changing landscape
of the development world, it is rare to utilize a single version of
language runtime. Even when you want to upgrade from one to the other
you'll need both usable on your system at the same time.
I've used tools like <code>pyenv</code> and <code>nvm</code> in the past when I needed to change
versions depending on which project I'm contributing to. But with <code>asdf</code>
you have one tool to rule them all!
<h2>Getting Started</h2>
The first thing you need to do when working with <code>asdf</code> is grab the
plugins for the languages you are interested in working with. You can list
what plugins are available:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf plugin list all
golang *https://github.com/kennyp/asdf-golang.git
golangci-lint https://github.com/hypnoglow/asdf-golangci-lint.git
nodejs *https://github.com/asdf-vm/asdf-nodejs.git
poetry *https://github.com/asdf-community/asdf-poetry.git
python *https://github.com/danhper/asdf-python.git
yarn *https://github.com/twuni/asdf-yarn.git</code></pre></div>
On the left will be the name of the plugin and on the right will be the repository
where it lives. It'll me marked with an asterisk if you already have it installed.
To install a plugin you say <code>asdf plugin add &#x3C;plugin></code> to get it installed. You can
also provide the repository where you want it pulled from, for example:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf plugin add nodejs https://github.com/asdf-vm/asdf-nodejs.git
> asdf plugin add python https://github.com/danhper/asdf-python.git</code></pre></div>
This will not give you any version of those languages, it is only installing the
plugin that knows how to work with those languages. You are ready to pull down
any versions you want at that point:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf install nodejs 14.19.0
> asdf install python 3.9.10</code></pre></div>
Once you have the versions installed you will be able to view them like this:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf list
golang
 1.17.7
nodejs
 --help
 12.22.10
 14.19.0
 16.14.0
 17.5.0
poetry
 1.1.13
python
 3.9.10
yarn
 1.22.17</code></pre></div>
<h2>Using the installed languages</h2>
To activate a specific version of a language you have you have three options:
<h3>Make it global</h3>
You can make it global, meaning when you run the tool like <code>python</code> it'll use
this version for the system:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf global python 3.9.10</code></pre></div>
<h3>Make it local</h3>
You can make it local, which means it will generate a file in the current
directory named <code>.tool-versions</code> and so whenever you change into a directory
it will activate the versions defined in there.
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf local nodejs 12.22.10
> cat .tool-versions 
nodejs 12.22.10</code></pre></div>
The great thing about this is you can commit that file to git and then anyone
who checks out the project and uses <code>asdf</code> will have the same versions activated!
<h3>Temporary</h3>
If you want to activate a version of a language temporarily you can swap to it
for the current shell:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf shell golang 1.17.7
> env|grep -i ASDF
ASDF_GOLANG_VERSION=1.17.7</code></pre></div>
It sets an environment variable that will have preference over the file. If you
ever wonder what versions a directory is using you can run:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">> asdf current
golang ______ No version set. Run "asdf &#x3C;global|shell|local> golang &#x3C;version>"
nodejs 12.22.10 .tool-versions
poetry ______ No version set. Run "asdf &#x3C;global|shell|local> poetry &#x3C;version>"
python 3.9.10 .tool-versions
yarn 1.22.17 .tool-versions</code></pre></div>
<h2>Conclusion</h2>
<a href="https://asdf-vm.com/">asdf</a> is an AWESOME tool to utilize if you find yourself using many
different languages or many different versions of the same language. You should check it out
and see if it can improve your workflow.

Use asdf to manage Python, NodeJS, GoLang and more!

A fun interview question some developers like to ask is to have you
convert ascii characters to an integer without using built in methods
like string.atoi or int().
So using python the obvious ways to convert a string to an integer are
these:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">>>> int('1234')
1234
>>> import string
>>> string.atoi('1234')
1234</code></pre></div>
The interesting thing here is finding out where on the ascii character
table the number is. Luckily python has this already built in with the
ord method:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">>>> help(ord)
 
ord(...)
 ord(c) -> integer
 
 Return the integer ordinal of a one-character string.
 
>>> ord('1')
49
>>> ord('2')
50</code></pre></div>
You can see that the numbers are grouped together on the ascii table, so
you just have to grab '0' as the base and subtract the rest:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">>>> ord('1')-ord('0')
1</code></pre></div>
So if we have the string '1234', we can get each of the individual
numbers by looping over it:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">>>> num_string = '1234'
>>> num_list = []
>>> base = ord('0')
>>> for num in num_string:
... num_list.append(ord(num) - base)
...
>>> print num_list
[1, 2, 3, 4]</code></pre></div>
but now how to we combine all these together to get 1234? You can't
just add them up because you'll just get 1+2+3+4 = 10.
So, we have to get 1000 + 200 + 30 + 4, which is a simple problem to
solve. Its just number times 10 to the nth power, so the final solution
is:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">num = '1234'
new_num = 0
base = ord('0')
 
for i,n in enumerate(reversed(num)):
 new_num += (ord(n) - base) * (10**i)
 
print new_num</code></pre></div>
This code is a little verbose though, lets make it a dirty nasty one
liner!
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">>>> sum([(ord(n)-ord('0')) * (10 ** i) for i,n in enumerate(reversed('1234'))])
1234</code></pre></div>

Convert a string to an integer in Python

Caesar Cipher in Python

eksclustergames.com walk through!

Automate project workflows with the command runner Just!

This article will show you how to use libvirt to create base images that
can be uploaded to OpenStack.
<h1>Why would you want to do this?</h1>
Linux distributions like Fedora and Ubuntu already ship "cloud" images
and most providers also have their own custom images for you to use, but
I find it much more comforting to have full control of the software that
is installed and I like the ability to easily apply new security patches
to base images.
I wouldn't use images to replace config management (CM) with something
like <a href="http://www.saltstack.com/">Salt</a> or
<a href="http://www.ansible.com/">Ansible</a> but they are nice to give sane system
defaults in things like <code>grub.conf</code>, <code>sysctl.conf</code>, and shipping a Chef
or Salt agent so that your CM engine can communicate with your server
right away.
<h1>Setting up your environment</h1>
The first thing you need to do is get a minimal install disk for the
Linux distribution you want to use. I prefer using Fedora netinst disks
but another popular option is Ubuntu Server.
To get the latest Fedora here, you can choose "netinst" under Direct
Downloads: <a href="http://fedoraproject.org/en/get-fedora-all">http://fedoraproject.org/en/get-fedora-all</a>
To get the latest Ubuntu you can go here:
<a href="http://www.ubuntu.com/download/server">http://www.ubuntu.com/download/server</a>
Once you have acquired your distribution of choice you just need to
verify that you have <code>virt-install</code> and <code>virt-viewer</code> installed:
Fedora:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">yum install virt-install virt-viewer</code></pre></div>
Ubuntu:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">apt-get install virtinst virt-viewer</code></pre></div>
If you prefer a graphical user interface, you may use <code>virt-manager</code>
instead, but I try to keep everything in the CLI; that way it can be
repeated easily.
<h1>Preparing your disk</h1>
Now that you have a base ISO and the tools necessary, let's get started
by creating a disk to install the virtual server into. Resizing an image
isn't an impossible task but it is much easier to choose a reasonably
sized disk for the purpose it will be used for.
I primarily use 8 GB disks -- that way we can fit all the system
components required as well as our own web applications. Any large files
should be placed in a SAN or something like Dreamhost's dreamobjects.
The other big decision you must make upfront is what disk format you
want to use -- the trade-off is disk space vs performance. The two
primary formats are qcow2 (QEMU Copy on Write) and Raw. qcow2 is great
if you have limited disk space and don't want to allocate the full 8 GB
up front. Raw is preferred if you want the best performance.
If you choose qcow2, you'll also need to make sure you have <code>qemu-img</code>:
Fedora:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">yum install qemu-img</code></pre></div>
Ubuntu:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">apt-get install qemu-utils</code></pre></div>
Create a raw disk:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">fallocate -l 8192M server.img</code></pre></div>
Create a qcow2 disk:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">qemu-img create -f qcow2 server.qcow2 8G</code></pre></div>
<h1>Installing your distribution onto the disk</h1>
We will use the <code>virt-install</code> command to get the distribution installed
onto the disk image.
To install Fedora on a qcow2 disk image:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">virt-install --name base_server --ram 1024 --cdrom=./Fedora-20-x86_64-netinst.iso \
--disk path=./server.qcow2,format=qcow2</code></pre></div>
To install Ubuntu Server on a raw disk image:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">virt-install --name base_server --ram 1024 --cdrom=./ubuntu-12.04.4-server-amd64.iso \
--disk path=./server.img,format=raw</code></pre></div>
You should follow the standard install steps that you normally would
when setting up your distribution. But here are some tips for each:
Fedora:
<ul>
<li>Choose minimal install -- by default it selects "GNOME".</li>
</ul>
Ubuntu:
<ul>
<li>
Be sure to select OpenSSH server -- it won't install it by
default.
</li>
<li>
On Ubuntu 12.04, there is a bug that makes it hang after running
<code>fsck</code>. You will need to edit grub to get it to boot, hit _<a href="">e</a> at
the boot prompt and add "nomodeset" on the linux line. You will
know that you need to do this if your boot hangs on fsck:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">fsck from util-linux 2.20.1
/dev/mapper/ubuntu--vg-root: clean, 57106/441504 files, 286779/1764352 blocks
/dev/sda1: clean, 230/62248 files, 39833/248832 blocks</code></pre></div>
</li>
</ul>
<h1>Preparing image for openstack</h1>
To prepare a virtual machine for the cloud, you will need to install the
<code>cloud-init</code> package, which allows the cloud providers to inject certain
system settings when creating servers based on the image. These are
things like hostname and ssh keys.
On Fedora:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">yum install cloud-init</code></pre></div>
On Ubuntu:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">apt-get install cloud-init</code></pre></div>
Then you need to just configure <code>cloud-init</code> by editing
<code>/etc/cloud/cloud.cfg</code> and update the <code>datasources_list</code> section to
include EC2. OpenStack uses EC2 metadata for <code>cloud-init</code>.
You should also verify the user setting in this same config and define
the user you plan to use, it will be where the <code>authorized_keys</code> file is
setup for when the cloud provider injects your SSH key into the server.
<code>cloud-init</code> will not create the user for you, it will just assign the
SSH keypair and reset the password. So make sure the user defined in
<code>cloud.cfg</code> is also created on the system.
Once you have your <code>cloud-init</code> settings the way you want them, just
shutdown and run the <code>virt-sysprep</code> command.
On the guest machine:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">shutdown -h now</code></pre></div>
On the host machine:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">virt-sysprep -d base_server</code></pre></div>
<h1>Uploading your image to OpenStack</h1>
Using the glance API it is very straightforward to upload the image to
OpenStack. Just run the following command:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">glance image-create --name base_server --disk-format=qcow2 \
--container-format=bare --is-public=True --file server.qcow2 --progress</code></pre></div>
Once the image upload completes you will be able to use it immediately
within nova. You can reference it by name or by the id from [glance
image-list]{.title-ref}.
To create your first instance from the image:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">nova boot --flavor m1.tiny --image base_server --key-name devops \
--security-groups free_for_all test_server</code></pre></div>
Obviously the security groups, key name, and flavors are based on your
installation of OpenStack but can all easily be queried from the nova
API:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="bash" data-theme="default"><code data-language="bash" data-theme="default" style="display: grid;">nova flavor-list
nova secgroup-list
nova keypair-list</code></pre></div>
And you are done! You'll be able to re-use your new image as a base for
all new instances you spin up in openstack!

Preparing custom images for OpenStack

How to speak spanish like a colombian drug lord!

I have seen a lot of people switch over to using ipython, bpython, etc
to get auto-complete support without realizing that the standard
interpreter does have this functionality.
To enable auto-complete support in the python interpreter you need to
create a python startup file that enables readline support. A python
startup file is just a bunch of python code that gets executed at
startup of the interpreter. To do this you just setup PYTHONSTARTUP in
your ~/.bashrc and then create a ~/.pythonrc.py file:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">#.bashrc
PYTHONSTARTUP=~/.pythonrc.py
export PYTHONSTARTUP
 
#.pythonrc.py
try:
 import readline
except ImportError:
 print("Module readline not available.")
else:
 import rlcompleter
 readline.parse_and_bind("tab: complete")</code></pre></div>
Now when you are in python you have tab completion on importing, calling
methods on a module, etc.
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">>>> import o
object( oct( open( or ord( os</code></pre></div>
I always end up using the pretty print module for viewing long lists and
strings in the interpreter so I prefer to just use it by default:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;"># Enable Pretty Printing for stdout
import pprint
def my_displayhook(value):
 if value is not None:
 try:
 import __builtin__
 __builtin__._ = value
 except ImportError:
 __builtins__._ = value
 
 pprint.pprint(value)
 
sys.displayhook = my_displayhook</code></pre></div>
It is also very useful to be able to load up your favorite editor to
edit lines of code from the interpreter, you can do this by adding the
following into your ~/.pythonrc.py:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;">import os
import sys
from code import InteractiveConsole
from tempfile import mkstemp
 
EDITOR = os.environ.get('EDITOR', 'vi')
EDIT_CMD = '\e'
 
class EditableBufferInteractiveConsole(InteractiveConsole):
 def __init__(self, *args, **kwargs):
 self.last_buffer = [] # This holds the last executed statement
 InteractiveConsole.__init__(self, *args, **kwargs)
 
 def runsource(self, source, *args):
 self.last_buffer = [ source.encode('latin-1') ]
 return InteractiveConsole.runsource(self, source, *args)
 
 def raw_input(self, *args):
 line = InteractiveConsole.raw_input(self, *args)
 if line == EDIT_CMD:
 fd, tmpfl = mkstemp('.py')
 os.write(fd, b'\n'.join(self.last_buffer))
 os.close(fd)
 os.system('%s %s' % (EDITOR, tmpfl))
 line = open(tmpfl).read()
 os.unlink(tmpfl)
 tmpfl = ''
 lines = line.split( '\n' )
 for i in range(len(lines) - 1): self.push( lines[i] )
 line = lines[-1]
 return line
 
c = EditableBufferInteractiveConsole(locals=locals())
c.interact(banner='')
 
# Exit the Python shell on exiting the InteractiveConsole
sys.exit()</code></pre></div>
For Django developers when you load up the ./manage.py shell it is nice
to have access to all your models and settings for testing:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="python" data-theme="default"><code data-language="python" data-theme="default" style="display: grid;"># If we're working with a Django project, set up the environment
if 'DJANGO_SETTINGS_MODULE' in os.environ:
 from django.db.models.loading import get_models
 from django.test.client import Client
 from django.test.utils import setup_test_environment, teardown_test_environment
 from django.conf import settings as S
 
 class DjangoModels(object):
 """Loop through all the models in INSTALLED_APPS and import them."""
 def __init__(self):
 for m in get_models():
 setattr(self, m.__name__, m)
 
 A = DjangoModels()
 C = Client()</code></pre></div>
After these tweaks the python interpreter is a lot more powerful and you
really lose the need for the more interactive shells like ipython and
bpython. All of these settings work in both python2 and python3.
If you want to see my complete ~/.pythonrc.py you can get it on
<a href="https://github.com/sontek/dotfiles/blob/master/_pythonrc.py">github</a>

Tips and Tricks for the Python Interpreter

Turning Vim into a modern Python IDE

Writing tests for Pyramid and SQLAlchemy

Having every github action run on every pull request will end up slowing you
down and sometimes even discourage you from making changes. For example, if you
see an error in the <code>README.md</code> but you know you'll have to wait for the entire
test suite to run you may choose not to push the change.
<h1>Path Filtering</h1>
To address this problem, Github has a feature called <a href="https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#onpushpull_requestpull_request_targetpathspaths-ignore">path filtering</a>
which works really well! You can include and exclude paths with a simple syntax:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="yaml" data-theme="default"><code data-language="yaml" data-theme="default" style="display: grid;">on:
 pull_request:
 paths:
 - '*'
 - '*/**'
 - '!README.md'
 - '!.gitignore'</code></pre></div>
Which will run on any changes except for <code>README.md</code> and <code>.gitignore</code>. There is
even a simpler format to this by using <code>paths-ignore</code>:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="yaml" data-theme="default"><code data-language="yaml" data-theme="default" style="display: grid;">on:
 pull_request:
 paths-ignore:
 - 'README.md'
 - '.gitignore'</code></pre></div>
Although the path filtering feature of github works, I do not recommend using
it because it has some critical issues that I'll talk about below!
<h1>Branch Protection / Required Checks</h1>
Path Filtering has one major flaw which is that it skips the run of the job
completely which means if you are using <a href="https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches">branch protection</a>
your PR will be stuck in <code>pending</code> state.
To resolve this we will have to run the job, but skip the actions inside it that
are slow conditionally. There are a few opensource actions out there that make
this easy but the one I recommend is:
<a href="https://github.com/tj-actions/changed-files">https://github.com/tj-actions/changed-files</a>
It creates a lot of useful outputs you can use in your conditional out of the box
which makes your action easy to write.
<h1>Skipping a required check</h1>
So using <code>tj-actions/changed-files</code> to skip a required check, it will look very
similar to the path filtering from github. You first define what files you care
about:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="yaml" data-theme="default"><code data-language="yaml" data-theme="default" style="display: grid;">jobs:
 terraform-lint: 
 runs-on: ubuntu-22.04
 steps:
 - uses: actions/checkout@v3
 - name: Check if files changed
 id: changed-files-yaml
 uses: tj-actions/changed-files@v40
 with:
 files_yaml: |
 src:
 - '*'
 - '*/**'
 - '!README.md'
 - '!.gitignore'</code></pre></div>
The <code>src</code> is just a key selection I made here. You can have many groups of
files like <code>tests</code>, <code>docs</code>, etc. and conditionally do things if each of them
changed.
So from our example, this step will generate output we can use <code>steps.changed-files-yaml.outputs.src_any_changed</code>
that is either <code>true</code> or <code>false.</code>. To use this we can us the <code>if</code> conditional
block on our jobs.
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="yaml" data-theme="default"><code data-language="yaml" data-theme="default" style="display: grid;">steps:
 - if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
 run: |
 earthly \
 +ci-terraform-lint \
 --TERRAFORM_VERSION=${{ env.RTX_TERRAFORM_VERSION }}</code></pre></div>
This will then skip linting if we didn't change any terraform and the job will
be marked successful!
<center>
<img src="/images/posts/github_skip_required_checks/skipped_checks.png">
</center>
This can be confusing sometimes and the people you work with (or future you!)
may wonder why these critical checks are being skipped. I like to include
a message with the decision that was made so its obvious that we wanted to skip
them:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="yaml" data-theme="default"><code data-language="yaml" data-theme="default" style="display: grid;">- if: steps.changed-files-yaml.outputs.src_any_changed == 'false'
 run: |
 echo "No terraform files changed"</code></pre></div>
Which will output:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="" data-theme="default"><code data-language="" data-theme="default" style="display: grid;">✅ "No terraform files changed"</code></pre></div>
Right next to the skipped job which is exactly what we wanted to see! So the end
result would look something like this:
<div data-rehype-pretty-code-fragment=""><pre class="github-dark-dimmed" style="background-color: #22272e" tabindex="0" data-language="yaml" data-theme="default"><code data-language="yaml" data-theme="default" style="display: grid;">on:
 pull_request:
 
jobs:
 terraform-lint: 
 runs-on: ubuntu-22.04
 steps:
 - uses: actions/checkout@v3
 - name: Check if real files changed
 id: changed-files-yaml
 uses: tj-actions/changed-files@v40
 with:
 files_yaml: |
 src:
 - '*'
 - '*/**'
 - '!README.md'
 - '!.gitignore'
 
 - if: steps.changed-files-yaml.outputs.src_any_changed == 'true'
 run: |
 earthly \
 +ci-terraform-lint \
 --TERRAFORM_VERSION=${{ env.RTX_TERRAFORM_VERSION }}
 
 - if: steps.changed-files-yaml.outputs.src_any_changed == 'false'
 run: |
 echo "No terraform files changed"</code></pre></div>
<h1>Conclusion</h1>
Hopefully with this you can start speeding up your github actions and making the
developer experience enjoyable!

sontek.net

Speed up github actions with conditional jobs, even with required checks

Path Filtering