How to Optimize Docker Builds with Nexus OSS for Apt, Maven, Docker and NPM Dependencies

In my journey of containerising the build process for my Android app, I soon realised the need to efficiently manage dependencies from repositories like Apt, Maven, NPM and Docker. To enhance the speed and reliability of my builds, I decided to set up a local proxy cache using Nexus OSS and configured all my tools to utilise this cache for fetching dependencies.

Rather than going out to the internet and downloading packages on every build, they are now cached in Nexus on my local network and can be retrieved much faster. After a week of usage, you can see Nexus is caching some 3.1GB of packages. Assuming these would other need to be fetched on every image or Android build, and a typical download speed of 2.5MB/s this saves 20 minutes of elapsed time – on every build.

Setting Up Nexus OSS

I created a Nexus instance by adding another service to my Gitlab docker-compose stack.

version: '2.3'

services:
  nexus:
    restart: always
    image: sonatype/nexus3
    volumes:
        - /mnt/ssd/appdata/nexus:/nexus-data
    environment:
      - INSTALL4J_ADD_VM_PARAMS=-Xms1200m -Xmx1200m -XX:MaxDirectMemorySize=2g -Djava.util.prefs.userRoot=${NEXUS_DATA}/javaprefs
    networks:
      - default
    ports:
     - 6326:8081

Once that is running, get the generated admin password from the nexus-data directory and log in to the web portal.

I set up a storage quota, allocating 10GB of my SSD drive space to the blog store.

Next, we will set up Nexus OSS repositories to proxy the various type of registries and repositories like NPM, Maven, Docker and Apt.

Dockerfile Configuration for APT Nexus Repository

One of the crucial steps in optimising Docker builds with Nexus OSS is configuring your Dockerfile to use the local proxy cache. Here’s how you can do it:

1. Create an Apt Proxy Repository: Inside the Nexus OSS interface, navigate to the “Repositories” section. Click on “Create Repository” and select “Apt (proxy)” as the repository type. Configure the proxy repository with a name apt-archive, URL for the remote Debian repository (e.g., http://archive.ubuntu.com/ubuntu), and set the distribution to focal.
2. Create Another Apt Proxy Repository: Repeat the above step to create a second Apt proxy repository, this time for the http://security.ubuntu.com/ubuntu/ repository. IN the distribution field, enter focal-security

Next, we need to make use of these repositories in our linux distribution by modifying the /etc/apt/sources.list file to point to your Nexus OSS repository, instead of the regular ubuntu.com repositories.
You can edit the sources.list directly, or — if you are using Docker –, create a local file with the same name and copy it into your container at build time. This is crucial because it ensures that apt uses the cached dependencies from Nexus instead of directly fetching them from the internet.

Dockerfile

COPY sources.list /etc/apt/sources.list

/etc/apt/sources.list

deb http://tower.lan:6326/repository/apt-proxy/ focal main restricted
deb http://tower.lan:6326/repository/apt-proxy/ focal-updates main restricted
deb http://tower.lan:6326/repository/apt-proxy/ focal universe
deb http://tower.lan:6326/repository/apt-proxy/ focal-updates universe
deb http://tower.lan:6326/repository/apt-proxy/ focal multiverse
deb http://tower.lan:6326/repository/apt-proxy/ focal-updates multiverse
deb http://tower.lan:6326/repository/apt-proxy/  focal-backports main restricted universe multiverse
deb http://tower.lan:6326/repository/apt-security/ focal-security main restricted
deb http://tower.lan:6326/repository/apt-security/ focal-security universe
deb http://tower.lan:6326/repository/apt-security/ focal-security multiv

In your Dockerfile, This is crucial because it ensures that Docker uses the cached packages from Nexus instead of directly fetching them from the internet. Here’s an example snippet to add to your Dockerfile:

Configure NPM to Use Nexus OSS

To further optimize your build process, don’t forget about NPM dependencies. You can configure your NPM installations to utilize Nexus OSS as well. Run the following command to set your NPM registry to your Nexus OSS repository:

npm config set registry http://your-nexus-oss-url/repository/npm/

Alternatively, add registry = http://your-nexus-oss-url/repository/npm/ to your .npmrc file. Note that I am using the NPM group repository created in Nexus.

Now, NPM will fetch packages from Nexus, reducing download times and enhancing build efficiency.

Hosting Other Static Dependencies in Nexus Repositories

Apart from caching Apt, NPM and Maven repositories, you can also host other large, static dependencies in Nexus to speed up downloads. For instance, consider hosting the Gradle Wrapper (150MB) and Google Android Command Line tools locally:

1. Create Nexus Repositories: In Nexus OSS, create hosted repository for these dependencies and upload the necessary files.
2. Fetch the Artifact URL: Retrieve the artifact direct URL from Nexus Portal
3. Configure Your Builds: In your build scripts or Dockerfiles, replace the original URLs with the nexus URL of these dependencies. This ensures that your builds fetch these resources from your local cache, avoiding downloading them from the internet for every build.

A final note

Another option is to embed a baseline set of dependencies in the Docker build image itself. That way, when your build pipeline executes NPM install, only a small delta of packages need to be fetched. This can be used in addition to Pipeline directory caching, as is available in Gitlab CI/CD.

Conclusion

In conclusion, optimizing Docker builds with Nexus OSS for apt, Maven, NPM, and other static dependencies can significantly improve your development workflow. By setting up proxy repositories, configuring Dockerfiles, and leveraging Nexus as a caching proxy, you can save time and ensure smoother and faster builds. Happy optimizing!