Part 2: Image Build Deep Dive

1. Introduction

In the first article of this series, we explained the high-level architecture for building a standardized development environment based on the “configuration-driven” idea. This article will dive into the execution core of this architecture—the image build phase—and detail how we leverage Docker’s Multi-stage Builds strategy combined with dynamic script generation techniques to create a development environment image that is both feature-complete and highly optimized.

2. Application of Multi-stage Build Strategy

To achieve modularity, maintainability, and maximum efficiency in the build process, we adopted a finely layered multi-stage build strategy. The Dockerfile is divided into five independent, purpose-driven stages. This methodology brings the following key engineering advantages:

  • Reduced Final Image Size: Only the artifacts from the final stage are kept in the final image. Dependencies needed only during the build process, such as compilation tools, intermediate products, and temporary files, can be safely left behind in earlier stages, thus preventing a bloated final image.
  • Improved Build Cache Utilization: Docker caches the result of each build stage. By placing less frequently changed layers (like base OS packages) at the beginning and more frequently changed layers (like application code or SDKs) at the end, we can maximize cache utilization and significantly speed up subsequent builds.
  • Enhanced Maintainability & Clarity: Each stage has a single responsibility (e.g., installing base tools, installing the SDK), which makes the Dockerfile structure clearer and easier to understand and maintain.

2.1 Detailed Breakdown of Build Stages

Our Dockerfile is logically divided into the following five stages:

Stage 1: base - Base Environment Layer

This stage is responsible for defining the operating system base and performing initial package manager configuration.

  • Responsibilities:
    • Specify a stable base image (e.g., ubuntu:20.04).
    • Configure APT software sources (sources.list), which can be pointed to an internal mirror based on the configuration in common.env to accelerate downloads and adapt to offline environments.
    • Install a minimal set of core system dependencies.
  • Artifacts: An environment with a base operating system and a configured package manager.

Stage 2: tools - Common Tools Layer

This stage builds on the previous one by installing development tools that are common to all platforms and do not change frequently with the project.

  • Responsibilities:
    • Install standard development tools like git, vim, ssh, distcc.
    • Install a specific version of gcc (as shown in offline_toolchain/gcc/install_gcc.sh) as part of the base compiler or system toolchain.
  • Artifacts: An environment integrated with a common set of development tools. Since the versions of these tools are relatively stable, this stage has a very high cache hit rate.

Stage 3: sdk - Platform-Specific Software Development Kit Layer

This is the core stage where platform differences are manifested. It is responsible for installing the cross-compilation toolchain and SDK specific to the target hardware.

  • Responsibilities:
    • Inherit from the base stage.
    • Receive platform-specific variables (like SDK_URL) from the .env file via the ARG instruction.
    • Execute a script (e.g., install_sdk.sh_template) to download, extract, and install the platform-specific SDK.
  • Artifacts: An environment with cross-compilation capabilities for a specific platform.

Stage 4: config - Environment Configuration Layer

This stage is responsible for the final configuration and personalization of the development environment.

  • Responsibilities:
    • Configure git-lfs tracking rules (gitlfs_tracker.sh).
    • Generate and configure network proxy scripts based on a template (proxy.sh_template).
    • Execute other environment initialization scripts (configure_env.sh).
  • Artifacts: A fully configured development environment, ready for finalization.

Stage 5: final - Final Artifact Layer

This is the last stage, aiming to create a clean, lightweight, and ready-to-use final image.

  • Responsibilities:
    • COPY all necessary filesystem content from the previous stage (config).
    • Clean up all unnecessary build caches and temporary files.
    • Set the container’s default user, working directory (WORKDIR), and entry point (ENTRYPOINT).
  • Artifacts: The final dev-env:${PLATFORM} image distributed to developers.

3. Core Technique: Build-time Dynamic Script Generation

To enable the build logic to respond to external configurations, we extensively use a technique of generating scripts dynamically at build-time. The core of this is using template files (with a _template suffix) and environment variable injection.

3.1 Implementation Mechanism

The workflow of this mechanism is as follows:

  1. Define Template Files: In the codebase, we create shell script templates containing placeholders, such as install_sdk.sh_template. These placeholders typically follow the shell environment variable format, like ${PLATFORM_SDK_URL}.

    install_sdk.sh_template (Conceptual Example)

    #!/bin/bash
set -e

echo "Downloading SDK from: ${PLATFORM_SDK_URL}"
wget -q -O /opt/sdk.tar.gz "${PLATFORM_SDK_URL}"

echo "Installing SDK..."
tar -xzf /opt/sdk.tar.gz -C /opt/

Further setup steps...

  2. Environment Variable Injection: When executing the docker build command, the build-dev-env.sh script injects variables read from the .env file into the build environment using the --build-arg parameter.

  3. Dynamic Rendering and Execution: In a Dockerfile RUN instruction, we perform a simple rendering process to convert the template file into an executable script, which is then executed immediately.

    Dockerfile (Conceptual Example in Stage 3)

    ARG PLATFORM_SDK_URL

COPY docker/dev-env-clientside/stage_3_sdk/scripts/install_sdk.sh_template /tmp/

RUN envsubst < /tmp/install_sdk.sh_template > /tmp/install_sdk.sh && \
    chmod +x /tmp/install_sdk.sh && \
    /tmp/install_sdk.sh
    • We use the envsubst tool (a standard GNU Gettext utility), which replaces strings in the ${VAR} format in the input stream with the values of the current environment variables.
    • This renders the final install_sdk.sh script.
    • We then give it execute permissions and run it.

3.2 Advantages

This dynamic generation mechanism is a direct manifestation of the “configuration-driven” idea in the build process, and it brings significant benefits:

  • Logic Reuse: We can write logically identical template scripts for different platforms, achieving different behaviors simply through different configuration inputs. This avoids writing nearly identical scripts for each platform.
  • Readability and Security: Configuration values (like URLs, version numbers) are clearly kept in .env files instead of being hardcoded in the Dockerfile or scripts. This makes configuration review and modification easier and also avoids exposing sensitive information (like private repository addresses) directly in the Dockerfile.

4. Conclusion

By combining Docker’s multi-stage builds with build-time dynamic script generation, we have implemented a highly modular, cacheable, and dynamically configurable image build process. This process not only ensures that the build artifacts are lean and efficient but, more importantly, provides the entire standardized development environment system with powerful flexibility and extensibility.