Table of Contents

General structure of elegant software:

I suspect that there is a way to formalize the general theory of elegant software using category theory and a few other bits and bobs, but for now I'll start with some lists and expand this as needed.

I would prefer that this be treated more like a checklist, or even better, as a ToDo list.

Suggestions for improvements to this list can be sent to me via the Contact Me page.

Configuration file

  1. Resides in `/etc/<program>` - This aligns with the Filesystem Hierarchy Standard, and is familar to everyone.
  2. Has sample code in the repository within a `samples/<program>.conf.example` file.
  3. Configuration language is suitable for given problem, but default to Lisp for now. If you are absolutely allergic to Lisp then YAML will suffice.
  4. Has defaults specified. Final configuration is the unification of the two. How the actual unification occurs is program-dependent, but the documentation and the sample configuration file should make it clear how the actual unification process occurs (do specific values override and take precedence, etc.)

Logging

  1. Logs to stderr, stdout by default
  2. If it is a non-trivial program, also sends logs to the systemd journal with a suitable unit name
  3. Accepts a `–quiet` parameter to inhibit stdout,stderr logging, but continues journal logging.
  4. Accepts a `–verbose` parameter to increase logging intensity
  5. May accept a `–debug` paramter to further increase logging intensity
  6. If journal logging isn't available (such as in some containers) then log to `/var/log/<program>` and logrotate that file on startup.
  7. Writes its process ID out to a given PID file in its state directory
  8. If you are running on a system without systemd-journald, then use the syslog facility for logging instead.
  9. In case of a distributed system, the system logging is configured for remote logging and logs are centralized somewhere that they can be analyzed separately.

Infrastructure

  1. Infrastructure is specified via code (IAC) and deployed via CICD pipeline with minimal manual steps (except for initial bootstrap).
  2. There should be at least two environments: A development environment, and a Primary environment (production)
  3. The differences between the environments are minimal, such that the exact same code instantiates all resources within the two, and the only possible difference the code can introduve (ideally) is in the names of resources.
  4. It should be possible to completely disintegrate the infrastructure and recreate it entirely from scratch without any data loss, and this has actually been done to test this.

Backups

  1. Data is periodically backed up via some external automation.
  2. Backups are triple replicated across sites.
  3. At least one of the backup copies is pull-based, not push, so that it cannot be destroyed via malicious actor gaining control of the main system.
  4. Backups have actually been tested regularly for restore-ability
  5. As a part of the CICD pipeline, some minimal backup must be restored to some ephemeral environment as one of the integration tests.

Parameters

  1. Mandatory to accept: `–config, –debug, –verbose, –quiet, –log-file, –pid-file`
Parameter Option What It Should Do
–config Allows overriding or specifying default configuration file path
–verbose Slightly increases logging output
–debug Significantly increases logging output
–trace Massively increases logging output
–quiet Inhibits all logging output except for the journal log
–log-file Allows specifying additional log file
–pid-file Allows specifying PID file location

Documentation

  1. Versioned in the repository together with the code to avoid drift.
  2. Wherever possible, autodocumenting tools or literate programming should be used so that the documentation is literally generated from and alongside the program during compilation.
  3. If software is to be run on a POSIX system, generating manpages in their standardized format are mandatory.
  4. If possible, generate info documents as well. (Optional)
  5. Where PDFs or other papers are required, they must be generated with something like LaTeX, and versioned alongside the code.
  6. Where graphs and other such diagrams are used, they must be generated via graphviz or other such tools, and the code versioned in the repository.

Monitoring and Alerting

  1. If software is not peformance critical, it records interesting telemetry about its internal state and how often different components are called. This will be useful for analysis later.
  2. Software makes available a prometheus-scrapable endpoint holding this telemetry, plus uptime.
  3. Some dashboards exist with both fine-grained detailed data, for debugging, and long-term generalized data (for trend analysis).
  4. Someone actually looks at the dashboards periodically.
  5. Every chart on the dashboard has at least two alerts set: One for No Data, and another for when the data is in a dangerous/suspicious range.

Deployment

  1. All deployment happens via the pipeline. This eliminates most of the cause of manual errors.
  2. An emergency break-glass procedure is in place for manual deployment in case of an emergency, but it is rarely used.
  3. Ideally, deployment occurs off of main branch on every commit that successfully passes all tests.
  4. ALL deployments MUSt go through the pipeline. If there is some process that blocks deployment (human review, etc.) then consider making that a part of the pipeline (pipeline job that requires human sign-off, etc). There is no process that cannot be codified formally, none. If lawyers can do it, so can we.

Licensing

  1. GPL-compatible license, or AGPL-compatible
  2. If software is meant for commercial use, then it must be dual-licensed, with a free software version available alongside the commercially licensed version.

Testing and Quality Gates

  1. Unit tests exist in a format suitable for the language at hand.
  2. Integration tests exist in a format suitable for the architecture and tools in use.
  3. On every commit, the pipeline runs unit tests. Merging to main is prohibited without all tests passing.
  4. Before the pipeline fully begins, sanity checks are made to ensure that nothing is amiss.
  5. After any deployment, sanity checks are made to ensure that the process succeeded.
  6. Unit tests should have at least 75% code coverage, measured using whatever tool is appropriate.

Heartbeats and Healthchecks

  1. Programs should have a periodic heartbeat where they register their 'aliveness' to some external service. In this way, if the program crashes, it is possible to know at least the last time that it was running.
  2. Periodic Healthchecks should also be made to the progam
  3. Healthchecks should actually check functionality of the program: Literally attempt to call some functionality and run sanity checks on the output.

Linting

  1. Maximum level of warnings and linting enabled
  2. Must be a part of the quality gates in the CICD pipeline.

Security

  1. Least privilege required for any and all tasks.
  2. Mandatory for all network connections to go over TLS.
  3. Public-key cryptography mandatory in the case of user authentication or service-to-service.
  4. Wherever possible, use infrastructure-based security - no keys, use the service accounts attached to the associated VMs or containers, etc.

Networking

  1. Must be IPv6-only with IPv4 at the edge.
  2. If interoperability with IPv4 is required, use DNS64 and NAT64. DO NOT INTRODUCE IPv4 into the internal network!
  3. Do not use DHCP6, allow all addresses to be dynamically generated via SLAAC.
  4. Do not allocate less than a /64 to a subnet, as this breaks SLAAC.
  5. Do not use IP addresses as stable identifiers, as they can and do change.
  6. Do not ever hardcode an IP address, anywhere. ALL code must use DNS names, with the one exception of any terraform code to initially assign DNS entries, which changes over time.
  7. If high-security environments are required, use either IPv6-allowlist-based security, or a point-to-point VPN on a separate IPv6 subnet or that traffic only.
  8. When using IPv6 private ranges, generate entirely random prefixes, do not use a prefix that anyone might conceivably share.
  9. When in the cloud, use either security groups or the equivalent to secure traffic via the native mechanism, do not roll your own firewalls unless you want headaches.

Packaging and Containerization

  1. Either Packaged at least for one major distro family: RPM or DEB, OR containerized.
  2. If packaged, the spec file must be part of the repository, all dependency versions must be pinned, and the build must proceed via the pipeline.
  3. If containerized, all dependencies must come from a local repository, NOT dockerhub or some public place (host your own dependencies!)
  4. If containerized, all dependency versions must be tagged and fixed.
  5. If containerized, the build must occur via the pipeline.
  6. All packages and containers must be signed upon building via PGP. Public key available on the project/owner's site.

Indentation

  1. Wherever possible, use tabs, not spaces. It is the user's choice how they wish to display their code, not yours. Tabs allow the user to configure how widely spaced their code is. We must always protect the user's freedom.
  2. Do not hard limit your lines to 80 characters, or to any length at all unless the language/tool requires it. It is the user's choice how they wish to display and format their text, and we must always protect the user's freedom.

Misc

  1. Compatible with the Red Hat / Fedora software packaging guidelines (has very strong requirements!)