SDK Modernization: From Monolith to `repo` & GitLab

Engineered the migration of a large, monolithic SDK from an offline tarball to a modular, multi-repository system on GitLab, managed by Google’s repo tool.

  • Author: Potter White
  • Date: August 2025

I. Project Overview

I independently spearheaded and executed the complete migration of a large-scale embedded Linux SDK. This project refactored a massive, monolithic codebase—originally distributed as an offline tarball—and migrated it to our internal GitLab server, establishing a modular, multi-repository collaborative development system based on Google’s repo tool.

This transformation not only introduced modern version control best practices but also solved critical technical challenges through an automation script. The result was a scalable, highly efficient team workflow that dramatically improved both development productivity and code quality.

II. The Challenge: “Tarball Hell” and the Pitfalls of a Monolithic Codebase

Before this initiative, our product development process, based on the Rockchip SDK, was plagued by severe challenges:

  • Lack of Version History: The SDK was distributed as a massive .tar.gz file, making it impossible to track changes. This plunged us into a “hell mode” of version management, relying on filenames like sdk_v1.2_final_final.tar.gz.
  • A Bloated Monolithic Repository:
    • Previously, the entire SDK was crammed into a single, enormous Git repository.
    • For instance, with the Rockchip SDK, official repo access requires a formal application process submitted to Rockchip’s representatives, including providing SSH public keys for authorized personnel. Without this, teams typically receive the SDK as a large, opaque tarball.
    • Consequently, cloning the SDK was incredibly time-consuming, merge conflicts were catastrophic, and developers working on different modules could not work independently. This forced a workflow where only one or two engineers could manage a project to avoid the immense overhead of code synchronization—a major bottleneck, especially when frequently modifying the kernel, device trees, or U-Boot.
  • Collaboration Gridlock: Sharing code changes depended on manual packaging and transfers, a process prone to errors. Modern collaborative practices like feature branches and Merge Requests were impossible, leading to integration nightmares.
  • Chaotic Access Control: All developers had access to the entire codebase, including sensitive IP and components outside their scope, posing a security risk.
  • Difficult Onboarding: Setting up a new development environment was a complex, multi-hour manual task that often required assistance from senior engineers.

III. My Solution: A Phased Approach to Architecting a Modern Development System

I designed and executed a phased solution to systematically resolve all the aforementioned pain points.

Phase 1: Automated Parsing & Intelligent Migration

Instead of an inefficient and error-prone manual migration, I developed a powerful Python automation script as the cornerstone of the project. The script was engineered to:

  • Intelligently Parse Complex Manifests: Automatically read and parse the nested repo manifest (.xml) files within the original SDK to accurately identify all N (the number of sub-repos is typically large, otherwise repo would be unnecessary) independent code modules.
  • Automate GitLab Project Creation: Leverage the GitLab API and Personal Access Tokens (PATs) to automatically create a corresponding, independent Git repository for each module within a designated team group.
  • Diagnose and “Heal” Unhealthy Repositories: The script automatically detected a critical issue: many of the original repositories were “shallow clones” with incomplete Git history. My solution was to:
    • For repos with full history: Preserve all branches and tags and push them to GitLab.
    • For repos with incomplete history: Automatically re-initialize them as new Git repositories, creating a clean, complete “initial version” commit. This step was crucial for ensuring the success of subsequent repo sync operations.
  • Standardize Branching Conventions: Automatically rename inconsistent default branches (e.g., master, develop) across all N repositories to a unified main, laying a stable foundation for the new repo manifest.

Phase 2: Hardening the GitLab Server Environment

To support the new workflow, I performed critical server-side configurations on our GitLab instance:

  • Configured fine-grained protected branch rules that allowed the migration script to perform an initial force push while protecting the main branch from accidental overwrites during daily development.

Phase 3: Crafting the New “Single Source of Truth”

I created a new, independent repo manifest repository. The manifest.xml file in this repository became the SDK’s blueprint and sole entry point, precisely defining:

  • The repository URLs for all N sub-projects and their local checkout paths.
  • The remote URL pointing to our internal GitLab server.
  • Perfectly Replicated the Original Build Environment: By using the <linkfile> tag, the script accurately recreated all necessary build script symlinks (e.g., build.sh, envsetup.sh) in the workspace root, ensuring a seamless transition for the developers’ build experience.

Phase 4: Empowering the Team & Documenting Knowledge

A successful system must be usable. To that end, I authored two core documents:

  1. The Administrator’s Migration Manual: This very document, detailing the entire migration process to ensure reproducibility and future maintainability.
  2. The Developer’s Guide: A clear and concise onboarding document that guides any team member to deploy the full development environment with just two commands (repo init and repo sync), complete with a detailed Q&A section that addresses all issues I anticipated and solved during testing.

IV. Results & Value: From Chaos to Control

This project fundamentally transformed our development workflow, delivering significant value:

  • ✅ 95% Reduction in New Developer Onboarding Time: Setup time was cut from hours of manual labor to under 10 minutes of automated commands.
  • ✅ Achieved 100% Version Control Coverage: We moved from zero history to full, granular Git history, enabling modern practices like code reviews, feature branching, and bug hunting with git bisect.
  • ✅ Enabled True Parallel Development: Developers can now work on different modules independently without interference, drastically reducing the complexity and risk of merge conflicts.
  • ✅ Enhanced Code Security and IP Protection: Using GitLab’s permission model, we can now grant engineers access only to the modules they are responsible for, safeguarding our intellectual property.
  • ✅ Established a Scalable Architectural Foundation: The new modular architecture makes it trivial to independently upgrade or replace components (like the Kernel or U-Boot) in the future, building a solid foundation for the product’s long-term evolution.

V. Tech Stack

  • Version Control: Git, Google repo
  • CI/CD & DevOps: GitLab Server Administration (gitlab.rb)
  • Scripting & Automation: Python (requests, subprocess), Bash
  • Core Domains: Linux, Embedded SDKs (Rockchip)