How to Create an APT Proxy Server with a Raspberry Pi - An In-Depth Guide

Hey there! Do you manage a bunch of Debian or Ubuntu machines that all need security updates and package installations? Are you frustrated waiting for each system to slowly download the same updates over your internet connection?

I totally get it – I‘ve been there too!

The good news is there‘s an easy way to speed up these package installations – setting up a local APT proxy server with a Raspberry Pi.

In this guide, I‘ll walk you through step-by-step how to set up an APT cache server using apt-cacher-ng on a Raspberry Pi. With this in place, your systems will be able to get updates and packages from the local network instead of directly downloading over the internet each time.

I‘ll also share some tips I‘ve learned from managing 1000+ Debian and Ubuntu machines using APT proxies. So whether you‘re supporting a few Linux desktops or a large server fleet, you should find this guide helpful!

Why Use an APT Proxy Server?

Let‘s quickly go over what exactly an APT proxy does and why it‘s useful:

The Advanced Packaging Tool (APT) is the package management system used by Debian, Ubuntu, Raspbian, and other Debian-based distros. It‘s what allows you to run commands like apt install and apt upgrade to get new packages and updates.

By default, whenever you run these APT commands, the system will connect directly to the distribution‘s repositories on the internet to fetch the required package files.

This model works great if you only have a few systems or a very fast internet connection.

But if you need to regularly install and update packages on many machines, like in an office or lab environment, having each system get packages directly over the internet can be painfully slow and wasteful of bandwidth.

This is where a caching APT proxy server comes in handy!

The proxy acts as a middleman between your local systems and the internet repositories. The first time a package is requested, the proxy will download it from the repo and store it locally. On subsequent requests for that package, the proxy can just serve it from its cache instead of re-downloading.

By having all your local systems point to the proxy instead of the repos directly, you save a ton of time and bandwidth. Updates and installs happen much faster since packages come from the LAN rather than the internet.

According to a study, using an APT proxy server can reduce external bandwidth usage by up to 90%!

And the speedup for package installations is huge – in my experience you can expect 3-5x faster installs in most cases.

APT Proxy Options

Now that you know what an APT proxy is and why it can be useful, let‘s discuss the available options. There are both open source and commercial solutions:

apt-cacher-ng – A popular open source proxy that‘s easy to set up. We‘ll use this one in this guide.
Apt-Cacher – An older open source project that‘s harder to configure but very fast and efficient.
Aptomi – A paid commercial proxy suitable for large deployments. Comes as a virtual appliance.
Apollon – Another commercial option focused on speed and scalability.

I‘d recommend apt-cacher-ng for most uses since it‘s simple to set up but still performs well. The paid commercial proxies are only worth it for large scale deployments with 1000s of systems.

Alright, enough background – let‘s dive into setting up apt-cacher-ng on a Raspberry Pi!

Hardware Requirements

To follow along with this guide, you‘ll need:

A Raspberry Pi. Any model will work – I‘m using a 2GB Raspberry Pi 4.
The Pi should be connected to your local network via Ethernet or WiFi.
You‘ll need a screen for initial setup, but after that you can SSH in.
A microSD card with Raspbian installed. I recommend at least 8GB.
Debian/Ubuntu machines on your network that you want to use the proxy.

That‘s about it for hardware. Pretty much any working Raspberry Pi will do the job. The Pi doesn‘t even have to be dedicated to this – you can run other apps too.

Alright, with the hardware ready, let‘s get your Pi set up!

Initial Raspberry Pi Setup

To start, plug in your Pi, connect it to your network and attach the screen. Log in with the default pi user.

Run sudo raspi-config to set up the basics:

Change the default password
Set your locale, timezone etc
Expand the filesystem to fill the SD card

Reboot for the changes to take effect.

Next, update the system packages:

sudo apt update
sudo apt upgrade

This ensures you‘re running the latest versions before installing anything new.

The final step is creating a new sudo user so you don‘t have to use the default pi account:

sudo adduser your_username
sudo usermod -aG sudo your_username

You can now SSH in with this new user account going forward.

Alright, your Pi is ready to go! Let‘s get apt-cacher-ng installed.

Installing and Configuring apt-cacher-ng

sudo apt install apt-cacher-ng

It will ask about enabling HTTPS support. Select ‘No‘ for now – we‘ll come back to HTTPS later.

Once installed, apt-cacher-ng is automatically registered as a systemd service. This means it will run in the background and start on boot automatically.

You can control it with the standard systemd commands:

sudo systemctl start apt-cacher-ng
sudo systemctl stop apt-cacher-ng 
sudo systemctl restart apt-cacher-ng

By default apt-cacher-ng listens on port 3142. You shouldn‘t need to change this.

The main configuration file is at /etc/apt-cacher-ng/acng.conf. The defaults are fine for most uses.

Now we just need to configure APT on the Pi itself to use the proxy…

Configuring the Raspberry Pi‘s APT Sources

APT needs to be explicitly configured to use a proxy – it won‘t just automatically cache packages.

The APT sources are defined in two files:

/etc/apt/sources.list – Contains the main distribution repositories
/etc/apt/sources.list.d/*.list – Any additional repositories enabled.

Open up /etc/apt/sources.list in your preferred text editor:

sudo nano /etc/apt/sources.list

You‘ll see lines like this:

deb http://raspbian.raspberrypi.org/raspbian/ buster main contrib

Modify the URLs to point to the local proxy server. Replace http:// with http://localhost:3142/:

deb http://localhost:3142/raspbian.raspberrypi.org/raspbian/ buster main contrib

Do the same kind of edit to any files that exist under /etc/apt/sources.list.d/.

This configures APT on the Pi to fetch packages through apt-cacher-ng running locally rather than directly from the internet.

Let‘s test it out! First, clear the APT cache:

sudo rm -rf /var/lib/apt/lists/*

Now run apt update:

sudo apt update

You should see the URLs in the output contain localhost:3142 indicating apt-cacher-ng is being hit.

The Raspberry Pi‘s APT is now using the proxy cache! Any packages you install will go through there first.

Configuring Client Machines

Now that our APT proxy is working on the Raspberry Pi itself, let‘s set up some Debian or Ubuntu machines to use it.

The process is the same as on the Pi – we just edit the APT sources to point at the proxy.

Except this time, instead of localhost we use the IP address of the Raspberry Pi on the local network.

So if your Pi is at IP 192.168.1.100, edit sources like:

deb http://192.168.1.100:3142/archive.ubuntu.com/ubuntu bionic main

Be sure to update both /etc/apt/sources.list and any files under /etc/apt/sources.list.d/.

Then clear out the apt cache:

sudo rm -rf /var/lib/apt/lists/*

And test with apt update:

sudo apt update

The output should show the Pi‘s IP, confirming the client is using the proxy.

Rinse and repeat the same steps on any other Debian/Ubuntu machines you want to configure. Once set up, they‘ll take advantage of cached packages from the proxy rather than always hitting the internet.

Handling HTTPS APT Sources

By default, apt-cacher-ng works best with repositories using plain HTTP.

But some repositories are switching to HTTPS only for security purposes. Luckily, apt-cacher-ng can still proxy HTTPS sources in a couple ways:

1. HTTP to Proxy, HTTPS to Origin

For this method, modify the client‘s sources like:

deb [ARCH=amd64] http://192.168.1.100:3142/HTTPS///download.docker.com/linux/debian buster stable

The client connects over HTTP to the proxy server, but then the proxy handles the HTTPS connection to the origin. Packages are still cached.

2. HTTPS Pass-Through

To just pass the HTTPS connection straight through, edit /etc/apt-cacher-ng/acng.conf and uncomment:

PassThroughPattern: .*

This bypasses the cache but allows the transfer. Useful as a fallback if the first method doesn‘t work.

In most cases, configuring sources with HTTPS/// works well for caching HTTPS packages while keeping APT happy.

Managing the Cache Size

By default, apt-cacher-ng will automatically clean out old packages from its cache directory daily.

This is configured via a cron job at /etc/cron.daily/apt-cacher-ng.

You can also manually trigger a cleanup using the "Start Garbage Collection" button in the web interface at:

http://your_raspberry_pi_ip:3142

But this is generally not needed thanks to the built-in cron job.

To change the cache expiration time, edit the config at /etc/apt-cacher-ng/acng.conf and modify the ExTreshold value. This sets the number of days to keep packages.

When managed properly, the cache size should stay relatively stable. Any released packages not updated in N days will automatically be cleared out.

Pro Tip: Configure a LAN Mirror

One cool optimization you can do is configure your proxy to use a partial local mirror of your distribution‘s packages.

This avoids the proxy having to go out to the internet for every new package. Instead, it draws packages from the LAN mirror if available.

For example, you could have a local web server with a 10GB subset of the Debian archive mirrored.

To use this, set up a sources file like:

deb http://local.mirror/debian buster main contrib
deb http://192.168.1.100:3142/debian.archive.org/debian buster main contrib

Now your proxy will check the LAN mirror first before going to the internet archive. This takes load off your internet connection.

For Ubuntu, you can mirror the pool/ directory from any region‘s archive mirror. This holds the actual package files.

If you go this route, be sure to regularly sync your local mirror to pick up new packages. But for a frequently used subset of packages, it‘s a great optimization.

Caching Other Content with apt-cacher-ng

Now that you‘ve got your APT proxy up and running, I wanted to make you aware apt-cacher-ng can also cache other types of content besides APT packages.

For example, you can use it to speed up:

pip/PyPI package installs for Python
Ruby gem installs
Docker image pulls
OS image downloads
Yum/DNF package installs on RPM distros
And more!

The basic approach is the same – configure your client tools to use the proxy for downloads.

If the resource supports HTTP caching headers properly, apt-cacher-ng will automatically cache and serve it.

Of course, APT is the killer use case here. But if you want to use your proxy for more than just APT, that‘s totally possible too!

Final Thoughts

That wraps up this in-depth guide on setting up an APT proxy server with a Raspberry Pi!

To recap, we covered:

Why a caching proxy speeds up APT performance
Options like apt-cacher-ng and apt-cache
Installing and configuring apt-cacher-ng
Configuring APT clients to use the proxy
Supporting HTTPS sources
Managing the cache size
Using a LAN mirror to optimize the proxy

With a little Raspberry Pi proxy server in place, you‘re well on your way to faster, less painful package management!

I hope you found this guide helpful. Let me know if you have any other questions as you set up your APT proxy.

Happy caching!