[Quick Install] Setup Local Repository (apt-get mirror) Update Server - Ubuntu

Set up a local Ubuntu (apt) repository with apt-mirror

 

This tutorial will cover the setup of an ubuntu/debian repository mirror. This is quite handy if you have multiple machines in your network running ubuntu. You only have to update the mirror, and the rest downloads the packages from there, so instead of having to download it 6 times (for 6 computers) you only download it once. Saves you and the mirrorservers some bandwidth.
We will be using the apt-mirror tool to do all the magic.
Please note that all the command need to be executed as root unless otherwise noted.

Installing apt-mirror

If you are on a recent ubuntu/debian version (+lucid or +lenny) you can execute the following command to install apt-mirror:

apt-get install apt-mirror

Configuring apt-mirror

Apt-mirrors config is quite easy. Below you can find my config file with comments, please adapt where needed.
To edit:

vim /etc/apt/mirror.list

/etc/apt/mirror.list:

############# config ##################
#
# Default variables are good enough.
# set base_path    /var/spool/apt-mirror
#
# set mirror_path  $base_path/mirror
# set skel_path    $base_path/skel
# set var_path     $base_path/var
# set cleanscript $var_path/clean.sh
# set defaultarch  <running host architecture>
# set postmirror_script $var_path/postmirror.sh
# set run_postmirror 0
set nthreads     20
set _tilde 0
#
############# end config ##############

# Start of repository URLs.

## Ubuntu Precise (12.04)
deb http://archive.ubuntu.com/ubuntu precise main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu precise-security main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu precise-updates main restricted universe multiverse

## Ubuntu Lucid (10.04 LTS) (Only 32 bit, no 64 bit machines running 10.04)
deb-i386 http://archive.ubuntu.com/ubuntu lucid main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu lucid-security main restricted universe multiverse
deb-i386 http://archive.ubuntu.com/ubuntu lucid-updates main restricted universe multiverse

## Debian Stable (atm 6, squeeze)
deb http://ftp.nl.debian.org/debian stable main contrib non-free

## Only 64-bit packages for debian testing
deb-amd64 http://ftp.nl.debian.org/debian testing main contrib non-free

## No backports needed for me.
## If needed, please uncomment.
#deb http://archive.ubuntu.com/ubuntu precise-proposed main restricted universe multiverse
#deb http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse

## Also no deb-src.  
#deb-src http://archive.ubuntu.com/ubuntu precise main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu precise-security main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu precise-updates main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu precise-proposed main restricted universe multiverse
#deb-src http://archive.ubuntu.com/ubuntu precise-backports main restricted universe multiverse

## I want to be able to downgrade the packages, so I don't want to clean up automatically.    
# clean http://archive.ubuntu.com/ubuntu
# clean http://ftp.nl.debian.org/debian

Downloading the packages

Now if the config is set up the way you want, we can start downloading the packages and filling the repository.
Execute the following command to start downloading. Note that this can take a while, here it took half an hour, so be patient. The packages will be stored in subdirectories of the /var/spool/apt-mirror/mirror directory, so make sure the partition that holds that directory has enough disk space left.

root@repo:/var/repo# su - apt-mirror -c apt-mirror

Output looks like this:

Downloading 14 index files using 14 threads...
Begin time: Mon Sep 17 14:26:32 2012
[14]... [13]... [12]... [11]... [10]... [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... [1]... [0]... 
End time: Mon Sep 17 14:26:37 2012

Proceed indexes: [PP]

149.1 GiB will be downloaded into archive.
Downloading 116234 archive files using 20 threads...
Begin time: Mon Sep 17 14:26:38 2012
[20]... [19]... [18]... [17]... [16]... [15]... [14]... [13]... [12]... [11]... [10]... [9]... [8]... [7]... [6]... [5]... [4]... [3]... [2]... [1]... [0]... 
End time: Mon Sep 17 15:03:22 2012

3.8 GiB in 5631 files and 1808 directories can be freed.
Run /var/spool/apt-mirror/var/clean.sh for this purpose.

Running the Post Mirror script ...
(/var/spool/apt-mirror/var/postmirror.sh)


Post Mirror script has completed. See above output for any possible errors.

Installing and configuring a web server

To be able to access the repo from other computers you need a webserver. You can also do it via ftp but I choose to use a webserver. You can install apache, lighttpd, cherokee, nginx or whatever suits you (boa, thttpd, awk-http).
Install a webserver (in this example, nginx)

apt-get install nginx

The packages are in /var/spool/apt-mirror/mirror. We are going to create a symlink to make them available. Make sure you edit the names and folders of the repo if your config is not the same as above.

ln -s /var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/ /var/www/ubuntu
ln -s /var/spool/apt-mirror/mirror/ftp.nl.debian.org/debian/ /var/www/debian

Your /var/www/ folder now should have these two items:

root@repo:~# ls -alh /var/www
lrwxrwxrwx  1 root root   54 Sep 17 15:54 debian -> /var/spool/apt-mirror/mirror/ftp.nl.debian.org/debian/
lrwxrwxrwx  1 root root   55 Sep 17 14:22 ubuntu -> /var/spool/apt-mirror/mirror/archive.ubuntu.com/ubuntu/    

Make sure you can see them via the webserver, go to the url of the server. If you've are doing the repo locally you can use this URL:

http://127.0.0.1/ubuntu

Note, for lighttpd you need to make sure that in the config the following option is enabled:

server.follow-symlink = enabled    

Setting up the clients

Now we have a working repo, we need to set up the clients to use this repo instead of the other (main) ubuntu repo's.
If you only have the standard repositories then you can use sed to comment all the default lines. If you are not sure, or have other repo's, please skip this step.
sed command to comment the entire /etc/apt/sources.list file:

sed -i 's/(.*)/#1/g' /etc/apt/sources.list /etc/apt/sources.list

Now edit the /etc/apt/sources.list file. Make sure to replace [[MYREPO]] with the http address of your repo (http://127.0.0.1 or http://myrepo.localdomain or whatever)

vim /etc/apt/sources.list 

And add the following (adapt if you have changed the config file):

deb http://[[MYREPO]]/ubuntu precise main restricted universe
deb http://[[MYREPO]]/ubuntu precise-updates main restricted universe
deb http://[[MYREPO]]/ubuntu precise-security main restricted universe multiverse

Or for debian:

deb http://[[MYREPO]]/debian stable main contrib non-free

Also comment out the other repositories (by adding a hash # before it).
Now reload the apt sources and see if it works:

root@repo:~# apt-get update
Ign http://127.0.0.1 precise InRelease
Get:1 http://127.0.0.1 precise Release.gpg [198 B]
Get:2 http://127.0.0.1 precise Release [49.6 kB]
Get:3 http://127.0.0.1 precise/main i386 Packages [1274 kB]
Ign http://127.0.0.1 precise/main TranslationIndex
Ign http://127.0.0.1 precise/main Translation-en 
Fetched 1324 kB in 0s (2009 kB/s)                
Reading package lists... Done

As you can see, it works!

Cleaning up

If the repo becomes to big, you can use apt-mirror to clean up some space. This will delete older packages. I choose to not do that automatically, so I can install earlier versions of packages, or downgrade existing ones.
Run the following command to clean up:

/var/spool/apt-mirror/var/clean.sh

Output may look like:

root@repo:~#     /var/spool/apt-mirror/var/clean.sh
Removing 5631 unnecessary files [4077010944 bytes]...
[0%]..................................................[8%]..................................................[17%]..................................................[26%]..................................................[35%]..................................................[44%]..................................................[53%]..................................................[62%]..................................................[71%]..................................................[79%]..................................................[88%]..................................................[97%]..............done.

Removing 1808 unnecessary directories...

===================================

Ubuntu Sources List Generator

With this we can generate ubuntu update source list



http://repogen.simplylinux.ch/   



Follow the above link



Troubleshooting Apt-get update (filed file not found)



try :



sudo apt-get clean
sudo rm -rf /var/lib/apt/lists/*
sudo rm -rf /var/lib/apt/lists/partial/*
sudo apt-get clean