Making your own Ubuntu Repository (BobSongs,etc)

HARDY HERON USERS
There’s a slight hiccup in the tutorial for Hardy Heron users. A solution has been found and works. If you’re using Gutsy Gibbon, Feisty Fawn, Edgy Eft or Dapper Drake, the solution does not apply to you.

Who Needs This Tutorial?
A local set of repositories proves useful for friends and family who have slow or no bandwidth. Assembled from various sources (see credits below) these steps should create ubuntu repository DVDs to share or keep in storage, however the need arises.

** IMPORTANT **
A complete repository download ties up the software servers; download rates will drop for daily users. If the download rate appears slow then pause the process and resume it (easily done, see the instructions below) at a later time.

Requirements (Please Read This)
Roughly 45 gigabytes free hard drive space: to store the complete set of .deb files Ubuntu offers; to create four DVD ISOs; left-over space for the function of the PC. (Upon downloading the repositories and creating the ISOs for Gutsy, my system says the folder contains: 72315 items, totaling 40 GB)

Index

1. Install the necessary tools covers what you’ll need to get started.
2. Extract debcopy is not necessary to do first, but we get it out of the way at the beginning.
3. The Big Download has the command necessary to download the desired repositories. Make your changes according to the type you need.
4. Divide into DVD-sized portions prepares the files to make the DVDs.
5. Create ISOs finishes what began in step 4.
6. Burning the ISO files covers how to write the files to blank discs.
7. Getting It All To Work is the instructions on how to tell Ubuntu to use the DVDs as a source for installing new software.
8. Updating the local repositories shows how to keep what you’ve downloaded up-to-date.
9. Poining Apt locally shows you how to use what you’ve downloaded to your own advantage (should you decide to keep it around).
10. Why not top it off with a setup DVD? is a link to Ubuntu setup DVDs for download.
11. FAQs answers some frequently asked questions.
12. Credits gives credit where credit is due.

1. Install the necessary tools
Open a terminal (From the top-left side of the screen: Applications → Accessories → Terminal) and keep it running throughout the length of this tutorial.

DAPPER, EDGY, FEISTY & GUTSY USERS
Copy the following code and paste it into the terminal. Hit the Enter key after pasting. (A quick way to “paste” text into the terminal is to hold down the following keys Ctrl + Shift + V).

Code:

sudo aptitude install debmirror ruby mkisofs debpartial dpkg-dev

HARDY HERON USERS (8.04)
OK. Here’s the solution. Although debpartial is not in the Hardy Heron repositories, it is still functional in Hardy Heron. Get the file from Canonical with this link:

CLICK HERE TO DOWNLOAD DEBPARTIAL

Once saved on the desktop double-click the file. Package Installer may tell you 2 new files are required to install debpartial:

• libruby
• libzlib-ruby

Accept the settings to proceed.

Now install the rest of the binaries/libraries you’ll need to proceed with this command:

Code:

sudo apt-get install debmirror liblockfile-simple-perl liblog-agent-perl ruby mkisofs dpkg-dev

2. Extract debcopy
The debcopy file is included in debpartial (downloaded and installed in Step 1). Though we will only need it later let’s extract it now. Please paste the following code into the Terminal:

Code:

cp /usr/share/doc/debpartial/examples/debcopy.gz ~

Code:

gunzip ~/debcopy.gz

3. The Big Download
The tricky part of this tutorial is the sheer number of choices here. That’s because five available Ubuntu distributions (Dapper Drake, Edgy Eft, Feisty Fawn, Gutsy Gibbon and Hardy Heron) work on four different kinds of computers: (i386, — your standard desktop PC, amd64, powerpc and sparc). This code is for the Hardy Heron (current version) repositories (version 8.04 on i386).

Code:

debmirror --nosource -m --passive --host=archive.ubuntu.com --root=ubuntu/ --method=ftp --progress --dist=hardy,hardy-security,hardy-updates,hardy-backports, --section=main,restricted,universe,multiverse --arch=i386 ~/UbuntuRepos --ignore-release-gpg

If you prefer another version or another kind of computer, CLICK HERE to choose among a very wide variety. I’d love to put them all here, but the tutorial would be painfully long.
Be prepared: this can take a long time, perhaps 24 or so hours. Downloading individual files instead of one large file takes longer.

Did the download stop halfway?
Press the up cursor key on your keyboard and the command should re-appear. Hit Enter and the download will pick up where it last left off.

Regular network glitches?
Try adding the following argument(s) to the command (the difference just being the number of seconds alloted before the retry begins) :

–timeout=seconds -t 120
–timeout=seconds -t 240

For more information on the command, click here. (Thanks to Jocose for this yummy tidbit!)

4. Divide into DVD-sized portions

• Note: this command may require editing:
• If you downloaded something other than Hardy Heron, replace hardy in the following code with the version name you downloaded in Step 3 (i.e., dapper, edgy, feisty, or gutsy).
  
• Replace –size=DVD with –size=CD74 (for 650 Mb CD-Rs), or –size=CD80 (for 700 Mb CD-Rs) if you plan on burning CDs instead of DVDs:
• Code

  debpartial --nosource --dirprefix=ubuntu --section=main,restricted,universe,multiverse --dist=hardy,hardy-security,hardy-updates,hardy-backports --size=DVD ~/UbuntuRepos ~/UbuntuDVDs

  Whether CDs or DVDs you need to determine the number of ISOs to create. Enter the following code in the Terminal:

  Code:

  ls -l ~/UbuntuDVDs

  If the last folder says ubuntu3 you need to create 4 DVDs. If ubuntu4 then it will be 5 DVDs, and if it says ubuntu31, you’ll need to create 32 CDs. Use the following codes to create each ISO:

Code:

ruby debcopy -l ~/UbuntuRepos ~/UbuntuDVDs/ubuntu0

Code:

ruby debcopy -l ~/UbuntuRepos ~/UbuntuDVDs/ubuntu1

Code:

ruby debcopy -l ~/UbuntuRepos ~/UbuntuDVDs/ubuntu2

Code:

ruby debcopy -l ~/UbuntuRepos ~/UbuntuDVDs/ubuntu3

Code:

ruby debcopy -l ~/UbuntuRepos ~/UbuntuDVDs/ubuntu4

Again: if creating CDs instead of DVDs (shudder) you may need to keep tweaking the code until you’ve done all 32 ISOs.

5. Create ISOs
These instructions assume you’ve got enough data to make 5 DVDs. If you’ve downloaded repositories that do not need a 5th DVD, none will be created.

• Instructions: Replace the following version number with what you downloaded. So replace 8.04 with 6.06 (if you downloaded dapper), 6.10 (for edgy), 7.04 (for feisty) and 7.10 (for gutsy).
• If you desire to make CD-R ISOs (detailed in Step 4) then make the following changes: replace -dvd1.iso with -cd1.iso, -dvd2.iso with -cd2.iso (and so on until all 30 or so CD-R ISOs are completed) :
  Code:

  mkisofs -f -J -r -V "Ubuntu 8.04 1/5" -o ubuntu-8.04-$(date -I)-complete-i386-dvd1.iso ~/UbuntuDVDs/ubuntu0

  Code:

  mkisofs -f -J -r -V "Ubuntu 8.04 2/5" -o ubuntu-8.04-$(date -I)-complete-i386-dvd2.iso ~/UbuntuDVDs/ubuntu1

  Code:

  mkisofs -f -J -r -V "Ubuntu 8.04 3/5" -o ubuntu-8.04-$(date -I)-complete-i386-dvd3.iso ~/UbuntuDVDs/ubuntu2

  Code:

  mkisofs -f -J -r -V "Ubuntu 8.04 4/5" -o ubuntu-8.04-$(date -I)-complete-i386-dvd4.iso ~/UbuntuDVDs/ubuntu3

  Code:

  mkisofs -f -J -r -V "Ubuntu 8.04 5/5" -o ubuntu-8.04-$(date -I)-complete-i386-dvd5.iso ~/UbuntuDVDs/ubuntu4

  6. Burning the ISO files

In Ubuntu

1. Insert a blank DVD (or CD) into your burner. A “CD/DVD Creator” or “Choose Disc Type” window will pop up. Close this, as we will not be using it.
2. Find the downloaded ISO image in the file browser (available at Places → Home menu on top of the screen.) Right click on the ISO image file and choose Write to Disc and wait for burning to complete.

In Kubuntu

1. Find the ISO image in the file browser (available at System Menu → Home Folder on bottom of the screen next to KMenu.)
2. Right click on the ISO → Actions → Write DVD (or CD) Image with K3b…
3. K3b will now automatically verify the md5sum.
4. Place Blank DVD (or CD) in burner and click on start.

In Xubuntu

1. Launch the burning tool, xfburn (available at Applications → Accessories → xfburn.)
2. Choose from the main toolbar, or from the menu Actions the option “Burn CD Image”,
3. From the Burn DVD (or CD) Image dialog box, click over (None) from “image to burn”, and find the ISO image in the file browser
4. In the dialog, click ‘Burn Image’.

Source.

7. Getting it all to work
There are two ways to add a DVD/CD to your repository list:

Synaptic:
Open Synaptic Package Manager (System → Administration → Synaptic Package Manager). Enter your password. From the menu: Settings → Repositories. Click the Third Party Software tab. Click the Add CD-ROM… button and follow the clearly written instructions in each dialog box. Repeat until all your media (DVDs, CDs) have been entered.

Terminal:
Open a Terminal (Applications → Accessories → Terminal) and enter the following command:

Code:

sudo apt-cdrom add

Insert the first disc and follow the instructions (it will ask you to name each disc). When apt-cdrom is finished with the disc, eject it with

Code:

eject

(yeah, just type it in and hit Enter) and insert the next and use the same command. Repeat the process per disc.

Once the last disc is done do this:

Code:

sudo apt-get update

followed by

Code:

sudo apt-get upgrade

which will update your system if any newer files are available. You can now open Synaptic Package Manager to use tens of thousands of packages.

(To be added: how to remove previously added DVD/CD entries in apt when old discs are replaced with new, as well as correcting any goofs that occur when incorrectly labeling the discs, etc.)

8. Updating the local repositories (i.e., “Now what!?”)
Now that you have gigabytes of .deb files all tucked neatly into the ~/UbuntuRepos folder… what on earth do you do to keep it updated? Start again? Sort of. Read on…

Open the Terminal (Applications → Accessories → Terminal) and go to the folder where you initially ran debmirror (by default: your home folder) and run the command in Step 3 again. This only makes an incremental update consuming roughly 10 minutes instead of 25 hours.

If new DVDs are required or desired, delete your original ISOs (in your home folder) and run steps 5 through 7 to recreate them after an update when necessary. Run this daily (as I do) and ISO creation won’t have the added bore of a long download.

9. Pointing Apt locally
If you keep this set of local repositories up to date and you’d like to use them to install software on your system, then use the following steps for lightning fast installations.

Apt relies on files called called Packages.gz to determine what is available to you. To create these open the terminal and do the following (this step takes a good 40 or so minutes, please be patient):

Code:

cd ~/UbuntuRepos && dpkg-scanpackages . /dev/null > Packages && gzip Packages && cd ~

Once completed insert the following into your sources.list file:

Code:

deb file:/home/[USER NAME HERE]/UbuntuRepos/ hardy main multiverse restricted universe
deb file:/home/[USER NAME HERE]/UbuntuRepos/ hardy-security main multiverse restricted universe

(Thanks, xfile087)

NOTE: Don’t forget these two things: 1) modify these lines for edgy, feisty and Gutsy, 2) put hash marks (#) before each Internet repository line you no longer require. Example:

Code:

# deb http://archive.ubuntu.com/ubuntu hardy-backports main restricted universe multiverse

10. Why not top it off with a setup DVD?
Head over to this link and select the corresponding Ubuntu suite you could include. Since bandwidth is pouring into your home like a waterfall, what’s a few more Mb, right?

__________________________________
FAQs

Q1 How would I do this through DOS/Windows?
A. I… honestly… don’t… know. I can’t imagine how tedious it would be to try to assemble all this file by file. If you’re a dual-booter or you’ve got faster bandwidth on a Windows machine elsewhere then I understand your dilemma. debmirror is the binary that does all the work. I cannot imagine that anyone has made a port for Windows. Until someone shows me a blog that tells Windows users how to accomplish this I suggest looking at the solutions in Q2.

__________________________________

Q2 Why don’t I simply head over to some FTP site to download pre-made DVD ISOs?
A1. Pre-made DVD ISOs are fine for a single, never-to-be-repeated download (if time is an issue). Here’s the difficulty: getting updates means downloading the ISOs again (sorta time consuming). If a single ISO download is right for you – use this link: ftp://ftp.leg.uct.ac.za/pub/linux/ub…-packages-dvd/

A2. Connecting through a phone line makes this tutorial impossible regardless the O/S. Consider purchasing inexpensive DVD ISOs from an online vendor. On-Disk.com and OSDisc.com provide exactly what you need: Ubuntu ISOs with setup disks – perfect for that off-line PC. These companies will ship these discs right to your door for a small fee (no affiliation with the author).

__________________________________

Q3 What if I wanted to combine or add in more repositories? Something like Canonical Commercial? Or PLF (now called MediBuntu)?
A. Yikes!! Don’t do it. Don’t do it!

debmirror will only erase your original files. You must change the name of the download folder(s). Let me explain.

This is what debmirror appears to do when run:

1. it receives a list of the files available over at the (newly selected) server
2. it compares it to what you have currently in your “destination” folder (in our case, /UbuntuRepos)
3. it erases everything in the destination folder that does not match the (newly selected) server
4. it begins the download.

If the user were to rework the original command in order to place the MediBuntu files in the ~/UbuntuRepos folder — in addition to what is already there — debmirror would clear ~/UbuntuRepos of anything that is not in the MediBuntu repositories (in other words: everything downloaded so far will be erased). Not recommended. Been there. Done that.

Rather than attempting to mix repositories to make a single DVD set: creating new folders to hold additional files is recommended. Thus you should create, say, ~/MediBuntuRepos and/or ~/CanonicalRepos for example. (Folder names may vary.) Then replace the ~/UbuntuRepos with ~/MediBuntuRepos and ~/CanonicalRepos in each command.

The following steps illustrate how to modify the original commands to download the Canonical Commercial and the MediBuntu repositories (both for Dapper Drake and i386). Their purpose is more educational than practical. By comparing the three different debmirror commands, as well as a little help from the man file and browsing the various source sites, you should be able to add any unusual repositories you may come across with little difficulty. Essential for knowing how to construct the command is visiting the site. For more details PM me.

Canonical (updated)

Code:

debmirror --nosource -m --passive --host=archive.canonical.com --root=/ --method=http --progress --dist=hardy,hardy-backports,hardy-proposed,hardy-security,hardy-updates --section=partner --arch=i386 ~/CanonicalRepos --ignore-release-gpg

Code:

debpartial --nosource --dirprefix=ubuntu --section=partner --dist=hardy,hardy-backports,hardy-proposed,hardy-security,hardy-updates --size=CD80 ~/CanonicalRepos ~/CanonicalRepos/CD

Code:

ruby debcopy -l ~/CanonicalRepos ~/CanonicalRepos/CD/ubuntu0

Code:

mkisofs -f -J -r -V "Canonical" -o ubuntu-7.10-$(date -I)-Canonical.iso ~/CanonicalRepos/CD/ubuntu0

Replace hardy with = dapper, edgy, feisty or gutsy
Architecture options = amd64, powerpc or sparc

MediBuntu (updated)

Code:

debmirror --nosource -m --passive --host=www.medibuntu.org --root=repo/ --method=http --progress --dist=hardy --section=free,non-free --arch=i386 ~/MediBuntuRepos --ignore-release-gpg

Code:

debpartial --nosource --dirprefix=ubuntu --section=free,non-free --dist=hardy --size=CD80 ~/MediBuntuRepos ~/MediBuntuRepos/CD

Code:

ruby debcopy -l ~/MediBuntuRepos ~/MediBuntuRepos/CD/ubuntu0

Code:

mkisofs -f -J -r -V "MediBuntu" -o ubuntu-7.10-$(date -I)-MediBuntu.iso ~/MediBuntuRepos/CD/ubuntu0

Replace hardy with dapper, edgy, feisty and gutsy
Architecture options = i386, amd64, powerpc

(To add MediBuntu to your personal repositories: https://help.ubuntu.com/community/Me…07a38a00d70e9f)

__________________________________

Credits
This tutorial was blatantly ripped off from Burt who admits he borrowed it from Ramon (original tutorial now here — originally designed for Hoary Hedgehog–somewhat out of date).

---