Skip to main content

The process of building software

The process of building software follows well defined steps:

  1. Source code and other artifacts are compiled and assembled into some sort of deployable artifacts. Primary tools in this spaces are 
    • Simple scripts
    • Make
    • IMake
    • Ant
    • Maven
    • Groovy
    • go build
    • cargo build
    • etc.
  2. Quality control steps are applied, either manual or automated:
    • Manual Testing
    • Unit Testing (JUnit, NUnit, cargo test, go test, etc.)
    • Integration Testing
    • System Testing
    • User Testing
    • Load and Performance Testing
    • Security Testing
    • Usability Testing
    • etc.
  3. The software is delivered.  This can take many different forms:
    • Manual installation
    • Automatic patching (Patch Tuesday)
    • Made available in a repository like Maven Central, docker hub, etc.
The major evolution in this process has not been the steps, but rather the boundaries between them.  When I first started, each and every step of this process was a manual handoff.  Now, large numbers of companies have software automatically migrating to a production environment if a change passes automated quality control steps.  And there is everything in between.

So how do you choose the right process for your environment?  Focus on the following principals:
  • Risk management - your process needs to reflect the risk you are taking with the change:
    • A failed simple A/B color scheme test for a GUI has a generally low risk so more automation is ok there.
    • A change to an API that transfers money using a banking API might find yourself on the front page of the newspaper.  In that case, having a human double check that the unit testing actually ran and reported correctly and that the correct banks where involved is totally reasonable.
  • Process and productivity - every manual step is one more place were having the wrong person on vacation or out sick can kill your productivity.  This is the inverse of risk.
    • For example, if every build requires that a DBA manually apply schema changes and run conversion scripts you will have lower cycle time than if that process is automated.
    • If you have automated migrations so that those same migration happen in the middle of a testing cycle, you will have invalidated that work.
  • Communication - every step needs to be visible to those involved.  This includes things like:
    • Build tags in source control - I've always used a variation on <major version>.<minor version>.<build number> where the build number increments for all builds that pass automated unit testing.
    • Build status - red, green, in-progress is the bare minimum.  A more feature full status will handle multiple parallel builds on the same component and show the current steps that are active and their outputs.
    • Feedback to change agents - this should be more than an e-mail to the development group, it should include project management, quality control, support staff and anyone else who impacts the build process.
In summary, treat your build process like a good group presentation: tell them what your going to build, build and test it, then tell them what the outcome was.

Comments

Post a Comment

Popular posts from this blog

Spring Boot native builds when internet downloads are blocked made simple

 No direct access to the internet If you work at a company that controls their software bill of materials, it's quite common to be blocked from directly downloading from: Maven Central Docker hub GitHub (the public parts) Getting the bits Maven Maven is first, because without it, you won't be able to compile your Spring Boot application, let alone move on to turning it into a native docker image. I will be showing changes need to work with artifactory, but you should be able to adapt it to other mirror solutions.  repositories {   maven {     name = "central"     url = "https://artifactory.example.com/central"     credentials {       username = "${project.ext.properties.artifactory_username}"       password = "${project.ext.properties.artifactory_apikey}"     }   } } With this configuration change, you should be able to download your plugins and dependencies, allowing you to compile and ...
Post a Comment
Read more

Kotlin Notebook when you're blocked from Maven Central

 TLDR; If you are blocked getting to maven central when first using Kotlin Notebooks because of company firewalls, you can use a tool like Fiddler Tool to redirect to a different network location. Kotlin Notebooks Kotlin Notebooks are a JDK based environment that brings the Python based Jupyter Notebooks  expressiveness to IntelliJ. From the blog post announcing the plugin, it looks like this: At home, the installation of jar files looked like this: I played around with it at home, but I couldn't use it at work.  Many companies, mine included, do not allow software components to be used when downloaded directly from the internet. In my companies case, we use a product called Artifactory, which allows you to mirror the content from Maven Central while still applying policies like CVE scanning, tracking, etc. The way it should work IntelliJ, as one of the leading IDE's, generally supports this quite well.  In fact, there is a whole setting page dedicated to dealing wi...
Post a Comment
Read more

Active vs. Passive Log4jShell remediation

 Log4jShell  All computer professionals should be aware of the Log4jShell ( CVE-2021-44228 ) and it follow on defects.  There is no shortage of opinions and lessons to be be learned: The difficulty of performing safe interpretation The problems when assumptions are not clearly documented.  I, for one, was completely shocked to find out that a logging system would actually attempt to do variable substitution in an actual message. The difficulty of finding and resolving issues with such a common library that is not provided by an OS package manager. IT'S A LOG4J CHRISTMAS One of my favorite podcasts, Security Now - episode 850 , discussed an analysis by Google regarding the depth of log4j dependencies.  From the show notes : One contributing reason is because Log4j is, more often than not, an indirect dependency. Java libraries are built by writing some code which uses functions from other Java libraries, which are built by writing some code which uses functions f...
Post a Comment
Read more