I was setting up a proxmox server at home (my “budget committee” approved the purchase) to replace a few aging servers:

FreeNAS1 2.5GHz Pentium DualCore 1GB RAM 5x 1.5TB
FreeNAS2 2.5GHz Pentium DualCore 2GB RAM 2x 500GB 2x 750GB
Proxmox1 2.6GHz AMD X2 4GB RAM 1x 160GB
Firewall 2.5GHz Pentium DualCore 2GB RAM 1x 160GB
Esxi 2.6GHz Pentium DualCore 8GB RAM 2x 1TB

With:

Prox1 Dual 2.4GHz Xeon 32GB RAM 2x 80GB 2.5″ SATA (non-Hotswap) 8x 2TB 3.5″ SATA in RAID50

I think the IPMI alone makes this worth while. Current power draw for the old machines is around 440 watts. Current draw for the replacement server is 415 watts, but that’s without much load. So power savings will be negligable.

So anyway, I had a couple issues with my idea that I needed to fix before making this production:
1. Proxmox uses GRUB bootloader, which means I cannot have a boot drive larger than 2TB in size. My fix was to install a pair of 2.5″ 80GB laptop drives to be the boot drives.
2. Proxmox will not install on a software or motherboard RAID from the standalone ISO. I set the BIOS to boot off the laptop drives only, same issues applied. I had to install proxmox with the RAID card removed from the system, and on a single card.
3. Fdisk does not support GPT. As old-school as I am, I guess I’ll have to ditch fdisk and learn parted.

I know I could have just carved out an additional LUN or even partitioned before installing proxmox to fix the first issue. But I wanted a single LUN for storage only, and it should be the entire 12TB raw. As far as the second issue, there is an install path that allows you to install debian before proxmox, which would then allow you to setup on the soft raid. I’m still tempted to go this route as it’s generally better to have two drives instead of just one. As I’m currently running proxmox 2.Beta, I think I still have a little time to decide.

**NOTE**
EXT3/4 supports up to 16TB of storage per partition!

List Current Mounted Partitions
df -h

List all disks available
fdisk -l

In my case I had the RAID listed as /dev/sda and the single boot drive as /dev/sdb so don’t get these confused!

Install Parted
apt-get install parted

Run Parted on the drive
parted /dev/sda

Change the filetype to GPT
mklabel gpt

Verify the disk geometry
print

Now I created 2x 6TB partitions to see how it would work
mkpart primary ext3 0 6000000
mkpart primary ext3 6000000 12000000
It looks like you can run mkpart primary ext3 0 -1 to utilize the entire drive

Quit parted
quit

Make the ext3 filesystem (with 1% reserved)
mkfs.ext3 -m1 /dev/sda1
mkfs.ext3 -m1 /dev/sda2

Make the mount directories
mkdir /mnt/sd1
mkdir /mnt/sd2

Mount the drives
mount /dev/sda1 /mnt/sd1
mount /dev/sda2 /mnt/sd2

Verify the drives are mounted
df -h
You should see /dev/sda1 and /dev/sda2 with 6TB each now

Add to fstab so they mount on boot
nano /etc/fstab

/dev/sda1 /mnt/sd1 ext3 defaults 0 0
/dev/sda2 /mnt/sd2 ext3 defaults 0 0

Save and quit

Reboot and verify all is well!

After playing around with ZFS for a while (it’s pretty nice, especially snapshots), I wanted to try non-solaris-type-deduplication. Opendedup was the answer.

I installed on a 2.8GHz Xeon Dell Poweredge 850 with 2GB RAM, and a mirrored array 500GB SATA drives.

Installed 10.04.3 LTS x64

Upgrade the Server
apt-get dist-upgrade
reboot

Install and Download
apt-get install openjdk-6-jre
apt-get install attr
wget http://opendedup.googlecode.com/files/sdfs-1.1.0_amd64.deb
wget http://opendedup.googlecode.com/files/sdfs_fuse_2.8.4.tar.gz
tar zxvf sdfs_fuse_2.8.4.tar.gz
cd sdfs_fuse_2.8.4
dpkg --install *.deb
apt-get install -f
dpkg -i sdfs-1.1.0_amd64.deb
apt-get install -f

Edit
nano /etc/security/limits.conf
Add the following above # End of file:

soft nofile 65535
hard nofile 65535

Create
/sbin/mkfs.sdfs --volume-name=VOLUMENAME --volume-capacity=CAPACITY
So in my case I wanted a 10GB volume for testing:
/sbin/mkfs.sdfs --volume-name=volume1 --volume-capacity=10GB

Mount
mkdir /mnt/volume1
/sbin/mount.sdfs -v VOLUMENAME -m MOUNTPOINT
/sbin/mount.sdfs -v volume1 -m /mnt/volume1

Verify
df -h

sdfs:/etc/sdfs/volume1-volume-cfg.xml:6442
10G 0 10G 0% /mnt/volume1

I then copied about 2GB worth of data. Looks like it parses through every 10 minutes or so – but since it’s the end of the day I’ll be checking in the morning.

There are a few common errors with simple fixes.
1. OutOfMemoryError – This is caused by the size of the DedupStorageEngine memory requirements being
larger than the heap size allocated for the JVM. To fix this edit the mount.sdfs script and increase the
-Xmx2g to something larger (e.g. -Xmx4g).

The files are actually stored in /opt/sdfs/volumes/VOLUMENAME
du -hc
Compare to df -h
df -h

In my example:
df -h
10G 5.4G 4.7G 54% /mnt/volume1
du -hc
3.1G total

So I’m using 5.4GB of space, but it’s only recorded as 3.1GB. Not a bad savings right there.

Next I’ll fill up the 10GB drive I created and see how much it really saves.

So I wanted a second volume shared for vmware using NFS. I ended up installing webmin and doing some of the configuration there for samba/cifs and nfs, but here’s my notes for commands on the opendedup server:

apt-get install nfs-kernel-server
nano /etc/exports
Add /export/volume2 *(insecure,no_subtree_check,rw,nohide,fsid=0)
/etc/init.d/nfs-kernel-server restart

Delete a Volume
If you ever need to delete a volume, make sure that you unmount it first:
umount /mnt/volume1
Then you can remove the data files:
rm -rf /opt/sdfs/volumes/volume1
The log files:
rm /var/log/sdfs/volume1*
And finally the configuration files:
rm /etc/sdfs/volume1*

Show Volume Info
/sbin/sdfscli --volume-info

I attempted to install OpenIndiana 151a on a Dell Poweredge 2650 server (2.8 xeon with 12gb ram, 5x 146SCSI, damn 3di controller). I put each disk into it’s own container on the controller and left the BIOS set to RAID.

I was receiving the following errors when attempting to install:

WARNING: Ignoring false simplex bit.
WARNING: Ignoring false simplex bit.
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: abort request, target=0 lun=0
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: abort device, target=0 lun=0
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: reset target, target=0 lun=0
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: reset bus, target=0 lun=0

IDE @ 0, the only IDE device I had was a CDROM drive. I figured it was a bad burn of the ISO – I verified the MD5 of the ISO I downloaded was correct, then I burned it again to CD (at 8X instead of 24X, just in case). Same issues persisted.

Looks like I need to disable DMA on the IDE channel to get this to work.
edit the grub configuration and add this at the end of the Kernel:
-B atapi-cd-dma-enabled=0,atapi-other-dma-enabled=0
Then run from that

I also ran the napp-it plugin:
wget -O - www.napp-it.org/amp | perl

Nexentastor Community edition did not want to run reliably on this hardware. I know it’s only 32-bit, so that is probably part of the issue. EON was just a giant pita for me, so I dropped that. FreeNAS was easy to setup but lacked on the ZFS support I wanted.

I don’t forget this, but I do have to teach people from time to time. Easiest way I know:
find / | grep FILENAMEYOUWANTTOFIND

so
find / | grep index.php

To find a file with certain words in the file:
grep -r ‘word or words you want to find’ /path/to/files/
grep -r ‘‘ /var/www/