Ubuntu Fresh Install

From Noah.org
Revision as of 02:20, 3 December 2014 by Root (Talk | contribs) (core packages)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search


This describes how I setup packages on my Ubuntu workstation after a fresh, clean install.

See also how I synchronize my home Dotfiles.

fresh install

This is my list of "must have" package I always install on a fresh Ubuntu machine. This is the minimal set of packages I use for a desktop development machine. This will get the machine ready for my typical work which includes basic sysadmin and Python development.

You can can also #transfer a list of packages from an existing Ubuntu installation.

core packages

These give me most of what I want from a general purpose box.

For lm-sensors (lmsensors, lm_sensors), install both lm-sensors and sensord.

The unifont package is a bitmapped font designed to cover as much of Unicode as possible in 8*16 and 16*16 pixel cells per glyph.

aptitude -q -y install vim-nox vim-scripts \
    openssh-server openssl lvm2 \
    nmap arp-scan iproute ipcalc bridge-utils vtun vlan \
    iw wireless-tools \
    bridge-utils uml-utilities \
    dmidecode i2c-tools lm-sensors sensord \
    rsync curl wget lynx w3m \
    bzip2 zip unzip arj bsdtar \
    clusterssh autossh screen dvtm dtach openvpn \
    git subversion cvs \
    gettext-base moreutils \
    libssl-dev libpcre3-dev libncurses-dev libreadline-dev libz-dev libusb-dev libzip-dev \
    libnet1-dev libpcap-dev pcaputils \
    libsqlite3-dev libcurl3-dev \
    xorg-dev libgtk2.0-dev \
    unifont unifont-bin \
    mesa-utils mesa-utils-extra libgl1-mesa-dev  \
    build-essential make autoconf autotools-dev libtool sed flex bison gawk \
    binutils sharutils patch diff patchutils \
    strace ltrace latrace \
    blktrace fatrace inotify-tools python-pyinotify \
    python python-dev python-setuptools python-pip python-profiler python-coverage pep8 \
    python-serial python-pyinotify python-levenshtein \
    python-tk python-imaging python-imaging-tk python-reportlab python-pydot \
    python-crypto python-protobuf python-plist python-pypcap python-geoip \
    pylint pychecker vim-python \
    plotutils gnuplot graphviz netpbm netpbm-dev libjpeg-dev libjpeg-progs \
    magemagick glv \
    linux-tools linux-base lttng-tools oprofile \
    libglapi-mesa \
    alsa-utils sox vorbis-tools mpg321 lame flac speex abcde id3v2 \
    apt-file debfoster apt-mark-sync add-apt-key \
    fuse-utils smartmontools samba samba-tools expect sux dialog \
    manpages-dev manpages-posix manpages-posix-dev linux-doc bash-doc sysadmin-guide \
    dc bc sqlite3 \
    prelink mkisofs bfr par \
    dstat sysstat iotop saidar statgrab adjtimex ips \
    tre-agrep safecat \
    cgroup-bin \
    xpdf-utils antiword enscript unhtml \
    mysql-server

Note that some hardware subsystems such as i2c (I2C), spi, and gpio have limited user-space tool support, but there are user-space interfaces to these buses under /sys/. For example, /sys/class/gpio. I should probably have a separate page for this information.

Network monitoring and diagnostics

aptitude -q -y install netdiag ifmetric ifenslave icmpinfo pcaputils \
    netdiscover imsniff tcpflow tcpick tcpslice tcpspy tcpstat tcptrace tcptrack tcpxtract \
    iw hostapd wpasupplicant ettercap kismet aircrack-ng wavemon swscanner \
    arping cutter dsniff macchanger \
    netcat socat \
    doscan nmap \
    slurm sniffit scanssh \
    nuttcp nfs-common nfswatch traceroute-nanog \
    arp-scan \
    mdns-scan nbtscan \
    avahi-utils avahi-discover python-avahi avahi-dnsconfd avahi-ui-utils \
    dnsmasq dnswalk dns-browse dnstracer dnstop fpdns pnscan \
    btscanner pads \
    nis netsed nethogs netmask \
    arpwatch arpalert farpd arpon

GUI and X11 apps

ttf-inconsolata is a good terminal and programming font.

aptitude -q -y install vim-gnome pidgin pidgin-otr xchat-gnome gitk \
    xterm aterm rxvt \
    inkscape xchm xclip \
    amarok k3b \
    compizconfig-settings-manager file-browser-applet \
    libnotify-bin \
    oggvideotools \
    gstreamer-tools python-gst \
    v4l2ucp libv4l-dev xserver-xorg-video-v4l v4l-conf \
    guvcview luvcview uvccapture \
    synergy xvfb xpra xnest xserver-xephyr \
    xserver-xorg-dev \
    ttf-inconsolata msttcorefonts

Extra apps

Less critical, but I like to have these around.

aptitude -q -y install \
    configure-debian swig conspy dconf cpu \
    mplayer mencoder ffmpeg x264 motion \
    enblend hugin jhead phatch zgv xzgv \
    cruft dcfldd foremost gpart iotop ips jablicator memlockd \
    csh memstat ncdu setserial statserial \
    graphviz python-pygraphviz quickplot \
    sc 

media, DVDs, video

For playing DVDs you need libdvdcss2. There are a few ways. On newer version of Ubuntu the easiest thing is to do the following. This will also install some other handy apps such as the Flash, unrar, and Java.

aptitude -q -y install ubuntu-restricted-extras

If that does not work for you then you can also try mediabuntu. That site will have info on adding a repository that provides libdvdcss2.

Save disk space

This generally doesn't matter on most machines, but the following tools can help on systems with limited space such as those with tiny SSD storage or embedded devices.

aptitude install -q -y localepurge

This tool can also help purge other noncritical files in order to free up space: Ubucleaner.

I also have a very old script called, deb_shrink.sh, that I used to use to shrink space for embedded Linux systems. This script has not been tested in a long time. It should be use more for ideas to start a new script. Click to download: deb_shrink.sh

Forensics

Note that the 'testdisk' package installs PhotoRec -- this is one of the better file recovery tools. Despite the name, PhotoRec is useful for recovering many types of files, not just photos.

aptitude install -q -y testdisk foremost scalpel chaosreader sleuthkit autopsy \
    recoverjpeg gpart ntfsprogs myrescue magicrescue dares sdd \
    dvdisaster dvdisaster-doc scanmem \
    rdd recoverdm rephrase \
    simhash

Security: IDS

aptitude install -q -y tct \
    tiger fail2ban

building and working with DEB packages

aptitude -q -y install build-dep devscripts fakeroot \
    autotools-dev dpatch po-debconf libpq-dev debhelper \
    debconf-utils \
    apt-dpkg-ref apt-doc \
    svn-buildpackage git-buildpackage \
    apt-file debfoster apt-mark-sync \
    pbuilder apt-move apt-src apt-listchanges \
    apt-transport-https debian-builder \
    debian-faq debian-reference debian-cd \
    debtags chkinstall debmirror

A typical workflow for building from deb source packages is to do the following:

apt-get source package-foo
cd package-foo-*
debuild

boot hacking

aptitude -q -y install pxe tftpd-hpa tftp-hpa python-tftpy kexec-tools pxe-kexec grub-ipxe ipxe ipxe-qemu kvm-ipxe tcpdump dhcpdump isc-dhcp-server isc-dhcp-client #isc-dhcp-relay

kernel hacking

This is incomplete.

See also Ubuntu Linux Kernel Compile.

aptitude -q -y install cramfsprogs squashfs-tools crash kexec-tools \
    build-essential autoconf autotools-dev libtool sed flex bison gettext \
    bin86 kernel-package libqt3-headers libqt3-mt-dev \
    git-core libncurses5 libncurses5-dev libelf-dev binutils-dev asciidoc \
    nfs-kernel-server tftpd-hpa msr-tools  \
    fakeroot kernel-wedge \
    linux-firmware-nonfree \
    linux-source \ # Includes Ubuntu's patches applied.
    linux-headers \ # Alternative to full linux-source if you are not rebuilding kernel.
    linux-tools \ # What the hell is in here? It looks like /usr/bin/perf_2.6.35-22 is the only thing.
    linux   linux-generic \  # What's the difference between these two?
    linux-doc

The official Ubuntu Linux Kernel build documentation shows this list:

sudo apt-get install fakeroot build-essential crash kexec-tools makedumpfile kernel-wedge
sudo apt-get build-dep linux
sudo apt-get install git-core libncurses5 libncurses5-dev libelf-dev asciidoc binutils-dev

Systems Development

aptitude -q -y install srecord statserial stressapptest superiotool ttylog

gripes

Ubuntu comes with a tiny vim, so you still have to install "vim" package to get the "real" vim. If you don't do this then the tiny vim will complain about most of the vim plugins you may install. The full-size Vim is not much bigger than the default tiny Vim! Vim-full should be the default. The tiny-vim package should be an option for tiny space-restricted installs.

The default python install is only missing one 15K package from the source installation! This is retarded! Basically they remove distutils. But anyone who works with Python knows that this is an essential package. Those who never look at Python and do not care will not miss the minisule amount of space that 'distutil' takes on the disk. This is a default package! They have to go through more trouble to remove it than it would have taken to simply not complicate matters with a separate 'python-dev' package.

Show list of packages explicitly installed by the user (no dependencies)

The first command searches the aptitude logs. The second command searches the apt logs. Note that zgrep works on uncompressed files as well as compressed files.

echo $(zgrep "\[INSTALL\]" /var/log/aptitude* | sed -e 's/^.*\[INSTALL\] \(.*\):.*/\1/'; zgrep -E '^Commandline:' /var/log/apt/history* | sed -e 's/^.* install \(.*\)/\1/')

This is an old way that gives only partial results

This will show a list of packages with their description explicitly installed by the user. See also dpkg --get-selections.

COLUMNS=299 dpkg -l "*" | grep ^i | sed -e 's/^[^[:space:]]\+[[:space:]]\+\([^[:space:]]\+\)[[:space:]]\+[^[:space:]]\+[[:space:]]\+[^[:space:]]\+[[:space:]]\+\(.*\)/\1, \2/'

transfer a list of packages from an existing Ubuntu installation

If you have not erased your exiting Ubuntu/Debian installation then you can get a list of all the packages that you have installed and then restore them when you install a new system. This does not save local configuration files (anything you might have edited in /etc).

This will save a list of the packages that have been installed. Note that this includes automatically installed packages to satisfy dependencies. This is not just a like of packages that have been explicitly installed.

dpkg --get-selections > dpkg.list

Use this to restore the packages:

apt-get update
dpkg --set-selections < dpkg.list
apt-get -u dselect-upgrade

Reconfigure console, environment, and Desktop

This fixes the entire machine. Or you can copy this file to ~/.local/share/applications/defaults.list and it will only effect your login session.

# Ubuntu 9.10
[ -f /etc/gnome/defaults.list ] &&  sed -i -e "s/gedit/gvim/" /etc/gnome/defaults.list
# Debian Lenny
[ -f /etc/gnome-vfs-2.0/defaults.list ] &&  sed -i -e "s/gedit/gvim/" /etc/gnome-vfs-2.0/defaults.list
gconftool-2 --dump / > gconf.dump

Big List of Package Favorites

Compression and archiving:

  • gzip | 295k | standard gz compression utility. Also handles .Z compress files.
  • zip | 254k | creates PKZIP compatible zip files.
  • unzip | 360k | reads PKZIP compatible zip files.
  • zipcmp | 77.8k | compare two zip files. ignores file order and compressed size differences.
  • bzip2 | 164k | compress and uncompress files in bzip2 format.
  • par2 | 279k | check and repair files in a par2 file set.
  • parchive | 111k | check and repair files in a par v1.0 file set.
  • uudeview | 135k |
  • unrar-free
  • unrar
  • sharutils | 999k | makes so-called shell archives out of many files, preparing them for transmission by electronic mail services.
  • pax | 152k | reads and writes tar and cpio formats, both traditional and extended specified in IEEE 1003.1.
  • bsdtar | 131k |
  • cabextract | 197k |
  • tofrodos | 81.9k | fromdos, todos, dos2unix, unix2dos
  • zoo | 156k | Zoo is used to create and maintain collections of files in compressed form.

Disk, CD, DVD, optical disc tools:

  • mdadm | 676k | used to create, manage, and monitor MD arrays for software RAID
  • fatresize | 69.6k | tool for non-destructive resizing of FAT16/FAT32 partitions.
  • vobcopy | 143k | mirror DVDs and copy VOB file.
  • cdck | 139k | creates table of read timing for each sector of a CD or DVD.
  • buffer | 81.9k | buffer streams for writing to disc.
  • dvdisaster | 1335k | Must be used BEFORE a disaster. DVDister creates error correction data which is used to recover unreadable sectors if the disc becomes damaged at a later time.
  • iat | 65.5k | tool for detecting the structure of many types of CD-ROM image file formats (BIN, MDF, PDI, CDI, NRG, and B5I) and converting them into ISO-9660.
  • bchunk | 73.7k | converts a CD image in a .bin/.cue format (sometimes .raw/.cue) into a set of .iso and .cdr/.wav tracks.
  • isomaster | 1081k | graphical CD image editor
  • macutils | 324k | utilities that deal with Macintosh files on a Unix system.
  • mtools | 475k | Tools for manipulating MSDOS disks.
  • mscompress | 81.9k | Microsoft "compress.exe/expand.exe" compatible (de)compressor

X11 x-windows tools:

  • unclutter | hide mouse pointer after mouse inactivity.
  • gnuplot-nox | 2,032k | plotting utility unrelated to GNU Plotutils.
  • gnuplot-x11 | 1,622k | plotting utility with x11 interface.
  • plotutils | 1,147k | GNU Plotutils -- plotting utility unrelated to gnuplot-nox.

Security:

  • wipe | 143k | repeatedly writes special patterns to the files to be destroyed, using the fsync() call and/or the O_SYNC bit to force disk access. Bullshit: http://abaababa.ouvaton.org/wipe/

Forensics:

  • unhide | 1655k | find processes and TCP/UDP ports hidden by rootkits, kernel modules, and by other techniques.
  • os-prober | 184k | detects other OSes available on a system.
  • disktype | 139k | detect the content format of a disk or disk image.

Video, UVC, V4L:

  • uvccapture | capture still JPEG frames from MJPEG video stream.
  • luvcview | view output of UVC device.

Laptops, Notebooks, Portables, Mobiles:

  • uswsusp: 496k | allows the system to have its state saved to disk and be powered off. Includes a program to suspend the system to RAM after the state is saved to disk. In that state, the system still uses power, but resuming is faster. If the battery depletes, the state is resumed from disk without data loss.

Development tools:

  • subversion | |
  • git-core | 14.6M | git main components with minimal dependencies.
  • git-svn | |
  • linux-doc | 7451k | kernel documentation. See /usr/share/doc/linux-doc/00-INDEX.
  • manpages-dev | 3269k |two sections: 2 = Linux system calls, 3 = Library calls
  • manpages-posix | |

packages under review

Debugging:

  • gdb
  • valgrind

Analysis:

  • tcpflow
  • arpwatch -- maintains a database of seen MAC addresses with associated IP pairs. Recommends mail-transport-agent.

Visibility:

  • smokeping
  • procps -- tools for looking at /proc virtual filesystem. Note name collision of `watch` command.
  • jnettop -- 143k | displays traffic streams sorted by bandwidth.
  • sysstat -- 1192k | performance analysis tools including sar, iostat, mpstat, pidstat, sadf
  • dstat -- 100k | dstat is performance analysis tool similar to sysstat.

Utility:

  • dsniff -- several tools to listen to and create network traffic: arpspoof, dnsspoof, dsniff, filesnarf, macof, mailsnarf, msgsnarf.
  • ettercap ettercap-gtk -- 438K 656K packet sniffer/injector for switched networks.
  • zerofree -- Fills unused blocks with zeros. Makes compressed disk images smaller. This is faster than using 'dd' to fill a dummy file with zeros.

Protection:

  • fail2ban -- I've been very happy with this package. It handle SSH as well as many other services.
  • sshguard

Audit:

  • ssldump
  • swscanner

Compliance:

  • sshfp -- generates RFC4255 SSHFP DNS records

Rejected:

  • flow-tools flowscan -- NetFlow clients and servers for Cisco NetFlow standard network monitoring.
  • snarf -- similar to wget, but smaller -- Meh...
  • lwatch -- like tail, but with colorize
  • ttysnoop -- This is a bit too heavy for ad-hoc tty sniffing. It might be good for certain types of debugging where you can setup a test environment ahead of time. This "snoops" on login tty's through another tty-device or pseudo-tty. A server must be configured ahead of time to close TTYs.
  • jpnevulator -- 86K | a serial sniffer