Skip to content

Latest commit

 

History

History
358 lines (268 loc) · 23.7 KB

README.md

File metadata and controls

358 lines (268 loc) · 23.7 KB
Raw

yplatform Github Actions CI Status

yplatform supports various cycles of software development, through a canonical set of commands.

yplatform aims to diminish cognitive load 💆, time to first build ⏱️ and time to first deployment 🚀 . Why?

yplatform was conceived within Tobii Pro's Cloud Services team 👋, previously known as support-firecloud .

GitHub GitHub contributors GitHub last commit (branch) GitHub commit activity (branch) GitHub language count GitHub top language GitHub code size in bytes GitHub repo size

Open in Gitpod

Table of Contents

Install standalone

git clone https://github.com/ysoftwareab/yplatform.git and you will gain access to:

  • a bootstrap process for local-development based on GNU and Homebrew
    • cross-platform.
      • No more worries about this is how it works on a developer machine (e.g. MacOS/Windows) and this is how it works in the CI (e.g. Linux).
    • brings common GNU utilities (bash, make, grep, sed, find, etc), version locked.
      • No more worries about different flags, different behaviour or missing features!
    • brings Homebrew.
      • No more worries about different versions available in distros' repositositories, or libraries/utilities not being available at all.
  • sane per-user configuration e.g. git
  • various utility scripts, 99% GNU Bourne Again Shell (Bash)
  • documentation and best common practices

Install in a project

bash -c "$(curl -qfsSL https://raw.githubusercontent.com/ysoftwareab/yplatform/master/bin/yp-install)" to install the latest yplatform (as a git submodule) to your project and you will gain access to:

  • a bootstrap process for CI machines based on GNU and Homebrew
    • cross-provider. No more worries about running pipelines with multiple CI providers, or switching to a new CI provider.
  • a build system based on GNU Make, both robust and flexible
  • sane per-repo configurations e.g. vscode
  • various utility scripts, 99% Bourne Again Shell (Bash)
  • a cloud infrastructure-as-code system based on GNU Make, both robust and flexible

Why?

yplatform aims to diminish cognitive load 💆, time to first build ⏱️ and time to first deployment 🚀 .

This means removing out-of-band information about system dependencies. How many times a repository's README mentions "something version x required"? How many times it doesn't mention anything, and you end up looking for a CI/Docker configuration?

This means removing differences in installing system dependencies. How many times did you look up "how to install X on Y?" How many times did you discover that you get different system dependencies, depending on the version of your Linux distribution?

This means removing differences in core utilities. How many times do you get something working on your MacOS development machine (BSD), only to see it fail on a CI machine, say running Ubuntu (GNU)?

This means removing differences in installing local dependencies. How many times a repository's README mentions npm install or yarn install or pip install or ./configure --prefix=/usr; make? Why should you care if this is a node/python/erlang/etc codebase, or what package manager it uses?

This means removing differences in triggering a build, checks, tests, a deployment, etc between codebases and their main programming language. How many times did you check how to build a codebase, what language it is using, what package manager or test framework it is using, what services need to be up, etc? How many times did you realize that checks are impossible to run locally, and that they run only in the CI, or worse - as a service reviewing/commenting your PR making it impossible to get early local feedback instead of feedback/noise in a Github pull request? How many times did you return years after to an old codebase, of your own above all!, only to realize you have no clue what's needed to build it again?

This means removing differences between what runs locally and what runs in the CI. How many times did you realize that you're instrumenting the CI ever-so-slightly-different than your local runs? Because maybe you duplicated code in the scripts section of your package.json and your GitHub workflow YAML?

This means removing differences between CI providers. How many times did you postpone trying out another CI provider? How many times did you choose a provider based on previous experience, popularity, how easy it is to integrate, rather than based on whether it is the most performant or the most appropriate for the job at hand?

This means consolidating best current practices. How many times did you look up how to set up encryption in a git repository? How many times did you reconfigure git with sane defaults or bump into CRLF/LF issues between system/developers? How many times did you configure or postpone altogether configuring a gitops flow? Env branches, tags, changelog creation, etc?

This means trying to make everything work the same everywhere. Emphasis on trying. In reality, 100% the same is not possible. But it is possible to achieve say 80%. And 80% is much better than 0% i.e. not trying at all.

Uniform interface

Starting fresh with a new project or switching between projects doesn't mean you need to switch development context, and instead you can focus on the business-logic context.

This mindset is useful for junior and senior developers alike. It makes onboarding new people as frictionless as possible.

Others have also been writing about this as early as 2016 and as late as 2021 and 2022.

Why use make you might ask. A few reasons actually:

  • make it's old (1976) but still under development, which makes it omnipresent, reliable and durable. In 5, 10, 20 years from now, expect it to still work!
  • lower cognitive load since chances are much higher that someone has already encountered make in other projects

The entrypoint to all projects that use yplatform is

Just type make !

make will fetch local dependencies, build the project and check that everything looks alright. That's all you need to have a fully functional development environment. Sometimes if your project has specific system dependencies, you may need to run make bootstrap.

"Then what?" you might ask.

Just type make help !

And you get a nice list of self-documented targets.

Want to see all targets? Type make help-all.

Want to fetch dependencies alone? Type make deps.

Want to build and skip dependencies and checks? Type make build.

What to run checks alone (i.e. linters)? Type make check.

Want to run tests? Type make test.

Want to release a new version as a git tag with automatic release notes based on git commits? Type make release.

Want to see commits since last release? Type make unreleased.

Want to promote a version to an environment branch? A branch that is coupled to a CI that deploys your software to a staging environment for example? Type make promote-env/staging/v1.2.3 for example.

Do you have secrets checked in and want to decrypt them transparently via git clean/smudge filters? Type make decrypt.

Do you want to reset your local git repository as if it was freshly cloned? Type make nuke.

And the list goes on and on and on. Fully extensible. Preloaded with minimal best current practices.

Cross-platform

Similarly yplatform itself runs across these platforms with very little effort and no duplication. Bootstrapped Docker images are also available.

Platform minimal common minimal (Docker) common (Docker)
Apple MacOS 11 ✔️ ✔️
Apple MacOS 12 ✔️ ✔️
Apple MacOS 13 ✔️ ✔️
Linux (Alpine 3.15) ✔️ ✔️ yp-alpine-3.15-minimal yp-alpine-3.15-common
Linux (Amazonlinux 2) ✔️ ✔️ yp-amzn-2-minimal yp-amzn-2-common
Linux (Archlinux) ✔️ ✔️ yp-arch-0-minimal yp-arch-0-common
Linux (Centos Stream 8) ✔️ ✔️ yp-centos-8-minimal yp-centos-8-common
Linux (Debian 9) ✔️ ✔️ yp-debian-9-minimal yp-debian-9-common
Linux (Debian 10) ✔️ ✔️ yp-debian-10-minimal yp-debian-10-common
Linux (RHEL 8.5) yp-rhel-8.5-minimal yp-rhel-8.5-common
Linux (Ubuntu 18.04) ✔️ ✔️ yp-ubuntu-18.04-minimal yp-ubuntu-18.04-common
Linux (Ubuntu 20.04) ✔️ ✔️ yp-ubuntu-20.04-minimal yp-ubuntu-20.04-common
Linux (Ubuntu 22.04) ✔️ ✔️ yp-ubuntu-22.04-minimal yp-ubuntu-22.04-common
Microsoft Windows 2019 (WSL) ✔️ 💤
Microsoft Windows 2022 (WSL) ✔️ 💤

NOTE RHEL needs entitlement certificates in the CI.

NOTE common install is paused for Microsoft Windows (WSL) because the process takes more than 1 hour on Github Actions CI (a combination of machine specs and being restricted to WSL v1), but we expect no problems.

NOTE it should even possible to create WSL images, and even OCI/podman images. Tracked in #213.

Cross-provider

yplatform itself currently runs across these providers with very little effort and no duplication.

Provider yplatform status master provider-branches version tags PR config env
AppVeyor AppVeyor Status ✔️ appveyor* config env
Bitrise Bitrise Status 💤 bitrise* config env
Buddy Buddy Status ✔️ buddy* config env
CircleCI CircleCI Status ✔️ circle* ✔️ ✔️ config env
Cirrus CI Cirrus CI Status ✔️ cirrus* ✔️ config env
Github Actions CI Github Actions CI Status ✔️ github* ✔️ ✔️ config env
Gitlab CI Gitlab CI Status 💤 gitlab* config env
Semaphore Semaphore Status ✔️ semaphore* ✔️ config env
Sourcehut Sourcehut Status ✔️ sourcehut* ✔️ config env

NOTE Bitrise builds for the master branch run only once a day due to freemium limits. NOTE Gitlab builds for the master branch run only once a week due to freemium limits.

We have also integrated in the past with more CI providers

Provider yplatform status master provider-branches version tags PR config env
Codeship Codeship Status 💤 codeship* config env
Drone CI Drone CI Status 💤 drone* config env
Travis CI Travis CI Status 💤 travis* config env

NOTE Codeship builds are paused because Cloudbees Codeship stopped offering their free plan. NOTE Drone CI requires explicit runners. NOTE Travis CI builds for the master branch are paused because one has to constantly ask for more open-source credits.

Show me!

Show me macOS and Ubuntu 20.04 bootstrapping

This is a bootstrapping of a fresh macOS developer machine.

Had to input my password (e.g. to install Homebrew) and decide post-installation if I optionally want to switch my shell from zsh to bash, something that is useful for those that are not techies. All done in 10 minutes. Set for life. Watch it at 10x, then it's only 1 minute. 😎

Second video is a bootstrapping of a fresh Ubuntu 20.04 developer machine. Had to input my password (e.g. to install some aptitude packages and Homebrew) and input my timezone. All done in 10 minutes. Set for life. Watch it at 10x, then it's only 1 minute. 😎

Show me ways-of-working with a project with yplatform

This is a cloning of a project on a macOS developer machine, and installing project-specific system dependencies.

Second video shows opening the same project, and playing around without actually having to know anything about it.

Repository structure

  • /bin has executable scripts, 99% GNU Bourne Again Shell (Bash)
  • /build.mk has build-system makefiles
  • /bootstrap has scripts that help bootstrap a machine
  • /ci has scripts that help steer the CI pipelines
  • /dev has scripts that help bootstrap a developer machine
  • /dockerfiles has scripts for the bootstrapped Docker images
  • /gitconfig has git configuration
  • /sshconfig has ssh configuration
  • /repo has configuration that is repo-specific, for those repositories bootstrapped with yplatform
  • /mk has common include makefiles e.g. sourced from /build.mk makefiles
  • /sh has common include shell scripts e.g. sourced from /bin shell scripts

Documentation

Writing

License

Apache-2.0