You are here: / neerv106.speed.planet.nl / serverconfig / hermes >

Hermes

(10-08-2005)   Growing tired of the lack of memory but excess of noise from the other Linux servers (henk, jan and harm), I decided to buy a single server to replace them all. The server would have to be -quiet -cheap -fast -cool (as in temperature) and have lots of memory.
(28-09-2005)   The generic kernel that comes with Slackware only uses the first gigabyte of RAM. After a bit of tinkering, it now uses all of it's memory.
(02-05-2006)   Extra disks added. I have decided to use LVM and mirrored storage. Yeah, I know, I'm a nerd.
(31-01-2009)   After seveal re-configurations, Hermes now has 3 disks: two 500GB PATA disks and one 300 GB SATA disk (which will soon be replaced by a 500GB SATA disk. Part of this storage will be used as storage for my Oracle RAC cluster using aoe.
The mirrored RAID-1 has been replaced by RAID-5 using partitions on all three disks.

Done:

  • NTP server/client
  • SETI@home client
  • DCHP server
  • Install & configure Apache 2.0
  • Configure MySQL
  • Migrate website from Henk
  • Kernel recompile (generic kernel won't do more than 1GB)
  • Migrate main storage (160 GB) from Jan to Hermes
Still to do:
  • Oracle 10g

Hardware

Model/form Tower (Selentium T2)
MotherboardGigabyte K7 Triton (GA-7VM400AMF) mATX
Processor AMD Athlon 2800+
Memory 2x1GB DDR400 Kingston ValueRam
Video integrated S3
Storage Maxtor 6L160P0 163"GB" (156334MB/8192KB)
HDS722516VLAAT80 164"GB" (157066MB)
Maxtor 6L300S0 300"GB" SATA (286188MB)

Lspci

The motherboard has integrated video, 6-channel audio and network (and AGP 8x, SATA, USB 2.0, a firewire slot and 2 DDR 400 memory slots), so there is no need for any PCI cards to install. Not having any of those cards reduces the amount of power and cooling needed and the amount of noise produced. I may eventually put in a video-capture card if I ever get around to wanting my own myth-tv-box, though.

00:00.0 Host bridge: VIA Technologies, Inc. VT8378 [KM400/A] Chipset Host Bridge
00:01.0 PCI bridge: VIA Technologies, Inc. VT8237 PCI Bridge
00:09.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05)
00:0a.0 Ethernet controller: Intel Corporation 82541PI Gigabit Ethernet Controller (rev 05)
00:0b.0 Multimedia video controller: Brooktree Corporation Bt878 Video Capture (rev 11)
00:0b.1 Multimedia controller: Brooktree Corporation Bt878 Audio Capture (rev 11)
00:0f.0 IDE interface: VIA Technologies, Inc. VIA VT6420 SATA RAID Controller (rev 80)
00:0f.1 IDE interface: VIA Technologies, Inc. VT82C586A/B/VT82C686/A/B/VT823x/A/C PIPC Bus Master IDE (rev 06)
00:10.0 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.1 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.2 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.3 USB Controller: VIA Technologies, Inc. VT82xxxxx UHCI USB 1.1 Controller (rev 81)
00:10.4 USB Controller: VIA Technologies, Inc. USB 2.0 (rev 86)
00:11.0 ISA bridge: VIA Technologies, Inc. VT8237 ISA bridge [KT600/K8T800/K8T890 South]
00:11.5 Multimedia audio controller: VIA Technologies, Inc. VT8233/A/8235/8237 AC97 Audio Controller (rev 60)
00:13.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL-8139/8139C/8139C+ (rev 10)
00:14.0 FireWire (IEEE 1394): Texas Instruments TSB43AB23 IEEE-1394a-2000 Controller (PHY/Link)
01:00.0 VGA compatible controller: VIA Technologies, Inc. VT8378 [S3 UniChrome] Integrated Video (rev 01)

Kernelconfig

Subject Module? Works? Kernelconfig
Memory yes <more..>
Network 8139too yes
Storage: USB stick sd_mod yes <more..>
Storage: SATA disk [scsi_mod sd_mod sata_via] yes (as builtin) <more..>
Storage: LVM [raid1], [raid456], dm_mod yes <more..>
Ext2/Ext3 filesystems jbd, ext3 yes <more..>
JFS filesystems jfs yes <more..>
DOS/FAT/NTFS filesystems msdos, vfat, ntfs yes <more..>
Video via, drm, via_agp, agpgart yes <more..>
Sound snd, snd_ac97_codeo, snd_mpu401uart, snd_page_alloc, snd_pcm, snd_rawmidi, snd_timer, snd_via82xx, soundcore yes <more..>
ACPI
Remote automount

Memory

(01-02-2009)   Hermes has 2 GB of RAM and is able to run every service I want run in my house (database, webserver, etc), without referencing swap. The default settings of kernel installed by slackware 11 only allows 1 GB of memory to actually be used.
I should be able to explain that, but my head hurts every time I try. Just turn on high-memory support and the kernel will be able to use the rest of the memory as well.

[Processor type and features]
  [High Memory Support]
    ( ) off
    (X) 4GB
    ( ) 64GB
  [ ] Allocate 3rd-level pagetables from highmem

Not sure about the 3rd-level thing. Since there's "only" 2 GB, I shouldn't really benefit from that option.

Storage

(01-02-2009)   This server now has 3 harddisks (2xPATA, 1xSATA) and must be able to access a USB stick (and a harddisk in USB enclosure). The three disks are managed through LVM, with a small mirrored and a large non-mirrored volumegroup. The root and temp filesystems are non-LVM. There are ext2, ext3 and jfs and even FAT filesystems involved.

Storage - USB

My USB stick and external USB-disk (normal PATA disk in USB enclosure) are seen as SCSI devices. Enabling SCSI disk support does the trick.

[Device Drivers]
  [SCSI device support]
    [M] SCSI device supports                 scsi_mod
      [M] SCSI disk support                  sd_mod
(fixme: what about

[Device Drivers]
  [USB Support]
    [*] Support for Host-side USB
      [M] USB Mass Storage Support            usb-storage
?)

Storage - S-ATA

One of the disks is a SATA disk, controlled through the libata library and seen as a SCSI disk. The SCSI subsystem is already included because of the USB stick, I "just" need to support the chipset.
Unfortunately, if I compile sata_via as a module, my SATA disk is not recognized at boot-time and the device-mapper doesn't see it ==> no (automatic) LVM. This is why the whole SCSI bit is compiled as a built-in.

[Device Drivers]
  [SCSI device support]
    [*] SCSI device supports                 scsi_mod
      [*] SCSI disk support                  sd_mod
      [SCSI low-level drivers
        [*] Serial ATA (SATA) support
          [*] VIA SATA support               sata_via

Storage - LVM

(01-02-2009)   Most of my diskspace is managed by LVM. I use two volumegroups: safevg, which is a volumegroup stored on mirrored disks and contains stuff I don't want to lose because of a single disk gone bad and datavg, for everything else.
For safevg, I use 3 partitions (1 on each disk), so choose RAID-5.
Enable "mirror target" so I can use pvmove

[Device Drivers]
  [Multi-device support (RAID and LVM)]
    [*]  RAID support
       [*]  Autodetect RAID arrays during kernel boot
       [*]  RAID-4/RAID-5/RAID-6 mode          raid456
          [*]  Support adding drives to a raid-5 array
    [M]  Device mapper support                 dm_mod
       [M]  Snapshot target
       [M]  Mirror target

Storage - Ext2/3

..fixme..

Storage - JFS

I use and like Ext3. Since I use LVM, I want to be able to resize filesystems. Ext3 lets me do that, but only by taking the filesystem offline. JFS allows me to resize (enlarge) a filesystem online. So:

[File systems]
  <*> JFS filesystem support             jfs
  [*]   JFS statistics
The driver can be compiled as a module, but since I'm not going to run the server without JFS, it's easier if it's compiled in.

Storage - FAT/NTFS

USB sticks/disks are usually formatted in some form of FAT/NTFS. Make sure the server knows how to read/write them:

[File systems]
  [DOS/FAT/NT Filesystems]
        MSDOS fs support                  msdos
        VFAT (Windows-95) fs support      vfat
     (437) Default codepage for FAT
     (iso8859-1) Default iocharset for FAT
        NTFS file system support          ntfs
     [*]   NTFS write support

Video

This motherboard has integrated video (S3 Unichrome Integrated Video, a VIA chipset). That means I won't be able to play Battlefield II and I also lose a bit of main memory to video, but since this is supposed to be a server and since there is 2 GB of memory anyway, that's not a problem. On the plus side, there is no add-on card (and noise and power) to worry about.
The idea is to get direct rendering support working on the chipset. Even though I'm not going to be using the video that much.

[Device Drivers]
  [Character devices]
    [M] /dev/agpgart (AGP Support)             agpgart
    [M] VIA chipset support
    [M] Direct Rendering Manager (XFree86 4.1.0 and higher DRI support)
    [M]   Via unichrome video cards            via
  [Graphics support]
    < > Support for frame buffer devices
    [Console display driver support]
      [*] Video mode selection support
Now, tell the X11 server to use this driver by adding a "Device" section in /etc/X11/xorg.conf and referencing it in the "Screen" section:
Section "Device"
    Identifier  "VIA chipset driver"
    Driver      "via"
EndSection

...

Section "Screen"
    Identifier  "Screen 1"
#    Device      "VESA Framebuffer"
    Device      "VIA chipset driver"
    Monitor     "My Monitor"

...

Sound

This is the current, working, config:
[Device Drivers]
  [Sound]
    [M] Sound card support
    [Advanced Linux Sound Architecture]
      [M] Advanced Linux Sound Architecture
      [ ]   Sequencer support
      [ ]   OSS Mixer API
      [ ]   OSS PCM (digital audio) API
      [PCI Devices]
        [M] VIA 82C686A/B, 8233/8235 AC97 Controller

Packages

These are the packages I have installed that are not part of the default Slackware installation. I try to find my software at www.linuxpackages.net in the form of a slackware package. If not there, I try to get a package from the source (opera). If there's no package, I get and compile source.

Project Name Found where Config Problems
Webserver Apache2 2.2.4 www.linuxpackages.net
Webserver PHP 5.2.1 for Apache 2 www.linuxpackages.net
Systemmonitoring Ifstatus 1.1.0 www.linuxpackages.net
CD ripping Lame 3.97b2 www.linuxpackages.net yes none
Players MPlayer 1.0pre8-i686-1jto www.linuxpackages.net none missing libs (libdivxdecore?)
Players MPlayer 1.0pre8-i486-1asz www.linuxpackages.net none missing libs (libdivxdecore?)
Players libdv-0.104-i486-1jed www.linuxpackages.net none
Players libtheora-1.0alpha5-i486-1jed www.linuxpackages.net none
Webcam xawtv-3.95-i486-2gds.tgz www.linuxpackages.net none
Camera digikam-0.9.0-i486-1kjz www.linuxpackages.net none
Camera Exiv2-0.12-i486-1kjz www.linuxpackages.net none
general hal-0.5.8a-i486-1kjz.tgz www.linuxpackages.net not yet used
general dbus-0.60-i486-2sdd.tgz www.linuxpackages.net not yet used

LVM

Hermes has 3 harddisks and while it would be easiest to just format them as 3 big ext3 filesystems and use them like that, it's not what I'm going to do. We're here to learn, right?
What I really want is: hassle-free boot, some mirrrored storage, a big chunk of freely allocatable disk space. The current situation is this (excuse the ASCII art):
physical device   hda              hdb        sda
device            hda1 hda2 hda3   hdb1       sda1
size (GB)         9    2    137    154        280
format            ext3 swap ext3   ext3       ext3

hda1    9  ext3  /
hda2    2  swap
hda3  137  ext3  /var/storage1
hdb1  154  ext3  /var/storage2
sda1  280  ext3  /var/storage3 (empty)
The root and swap partitions will stay the same, hdb is going to be split into a 137 GB partition and a 17 GB partition, so I can build a 137 GB RAID-1 volume. The rest will form a 297 GB non-mirrored volume:
physical device   hda             hdb        sda
device            hda1 hda2 hda3  hdb1 hdb2  sda1
size (GB)         9    2    137   137  17    280
RAID                        ----  ----
LVM                         <-safevg-> <-datavg->
format            ext3 swap

volume groups
  safevg, RAID-1, 137 GB
  datavg, non-mirror, 297 GB

logical volumes
  backuplv   safevg   10GB  ext3  /var/backup
  wwwlv      safevg   10GB  ext3  /var/www
  medialg    datavg  100GB  ext3  /var/media
  oralv      datavg   10GB  jfs   /oradata
  etc.
Of course, hda3 and hdb1 are currently > 90% full. There's a lot of data that needs to be moved back and forth..

Making datavg

Hdb2 won't become available until after hdb is refdisked, so datavg is initially going to consist of only sda1. Fdisk sda to have one partition of type 8e (Linux LVM).
  • Build your lvm database:
  • vgscan
    Reading all physical volumes. This may take a while...
    No volume groups found
  • Mark /dev/sda1 as a physical volume used for LVM
  • pvcreate /dev/sda1
    Physical volume "/dev/sda1" successfully created
  • And create a volume group based on this volume
  • vgcreate datavg /dev/sda1
    Volume group "datavg" successfully created
  • Create the first volume wwwlv of 5 GB
  • lvcreate -L 5G -n wwwlv datavg
    Logical volume "wwwlv" created
  • Create a filesystem on wwwlv and mount it
  • mke2fs -j -L 20060506 -m 2 -T news /dev/datavg/wwwlv
    mount /dev/datavg/wwwlv /var/www2
With LVM and JFS it's possible to create a filesystem and enlarge it on the fly (handy for those 24x7 databases that outgrow their storage).
  • Create a LV oralv, in which to store databasefiles
  • lvcreate -L2G -n oralv datavg
    Logical volume "oralv" created
  • Create a filesystem /oradata on this LV
  • jfs_mkfs -L 20061127 /dev/datavg/oralv
    jfs_mkfs version 1.1.11, 05-Jun-2006
    Warning! All data on device /dev/datavg/oralv will be lost!

    Continue? (Y/N) y
    |

    Format completed successfully.

    2097152 kilobytes total disk space.
  • Add a line in fstab and mount the filesystem (this is the 11th filesystem to be added during boot)
  • fstab: /dev/datavg/oralv /oradata jfs defaults 1 11
    mount /oradata
  • After a while, the new FS fills up and we need to enlarge it, but without stopping the database (or making the tablespace read only and moving the datafiles)(or creating new datafiles or .. oh, just shut up)
  • lvextend --size +2G /dev/datavg/oralv
  • The LV is extended, but the FS doesn't see it yet. Normally, the FS would have to be taken offline, then resized to fill the new space and then remounted, which means down-time for our database (more importantly: work for the DBA). But here's the JFS party-trick: it can do all that online:
  • mount -o remount,resize /oradata
Et voila, now /oradata is 4 GB big with no down-time!
See also http://www.slackersbible.org/book/print/3

Making safevg

Safevg is a volumegroup, based on a RAID-1 device, made up of a pair of 137 GB partitions. First, /dev/hda3 and /dev/hdb1 are created and set to type 'fd' (Linux raid autodetect - not strictly necessary). The two partitions are slightly different in size by a few KB, but this can't be helped. The rest of the second harddisk, a 17 GB partition by the name of /dev/hdb2, will later be allocated to datavg.
  • Create a RAID-1 device:
  • mdadm --create /dev/md0 --level=raid1 --raid-devices=2 /dev/hda3 /dev/hdb1
  • Show result:
  • mdadm --misc --detail /dev/md0
  • Make it an LVM device:
  • pvcreate /dev/md0
  • Create the new volumegroup
  • vgcreate /dev/safevg /dev/md0

Add hdb2 to datavg

A 17 GB part of /dev/hdb wasn't used for the RAID volume. I don't want to manage it separately, so make the partition of type 8e and add it to the datavg volumegroup.
  • Make /dev/hdb2 an LVM device:
  • pvcreate /dev/hdb2
  • Add it to datavg:
  • vgextend datavg /dev/hdb2

(01-02-2009)   Safevg is now a RAID-5 array with partitions from three disks. Creating a RAID-5 array is very similar to a creating a RAID-1 array:

  • # create RAID-5 device
    mdadm --create /dev/md0 --level=raid5 --raid-devices=3 /dev/hda9 /dev/hdb1 /dev/sda2
    mdadm --detail /dev/md0
    # create safevg on RAID-5 device
    pvcreate /dev/md0
    vgcreate /dev/safevg /dev/md0
    # create logical volumes
    lvcreate -L22G -n audiolv safevg
    # create filesystems
    jfs_mkfs -L20090127-1 /dev/safevg/audiolv
    # mount filesystems
    mount /var/media/audio
Back to top
@Hans © Copyright 2005-2009 Hans Neervoort