After playing around with ZFS for a while (it’s pretty nice, especially snapshots), I wanted to try non-solaris-type-deduplication. Opendedup was the answer.

I installed on a 2.8GHz Xeon Dell Poweredge 850 with 2GB RAM, and a mirrored array 500GB SATA drives.

Installed 10.04.3 LTS x64

Upgrade the Server
apt-get dist-upgrade
reboot

Install and Download
apt-get install openjdk-6-jre
apt-get install attr
wget http://opendedup.googlecode.com/files/sdfs-1.1.0_amd64.deb
wget http://opendedup.googlecode.com/files/sdfs_fuse_2.8.4.tar.gz
tar zxvf sdfs_fuse_2.8.4.tar.gz
cd sdfs_fuse_2.8.4
dpkg --install *.deb
apt-get install -f
dpkg -i sdfs-1.1.0_amd64.deb
apt-get install -f

Edit
nano /etc/security/limits.conf
Add the following above # End of file:

soft nofile 65535
hard nofile 65535

Create
/sbin/mkfs.sdfs --volume-name=VOLUMENAME --volume-capacity=CAPACITY
So in my case I wanted a 10GB volume for testing:
/sbin/mkfs.sdfs --volume-name=volume1 --volume-capacity=10GB

Mount
mkdir /mnt/volume1
/sbin/mount.sdfs -v VOLUMENAME -m MOUNTPOINT
/sbin/mount.sdfs -v volume1 -m /mnt/volume1

Verify
df -h

sdfs:/etc/sdfs/volume1-volume-cfg.xml:6442
10G 0 10G 0% /mnt/volume1

I then copied about 2GB worth of data. Looks like it parses through every 10 minutes or so – but since it’s the end of the day I’ll be checking in the morning.

There are a few common errors with simple fixes.
1. OutOfMemoryError – This is caused by the size of the DedupStorageEngine memory requirements being
larger than the heap size allocated for the JVM. To fix this edit the mount.sdfs script and increase the
-Xmx2g to something larger (e.g. -Xmx4g).

The files are actually stored in /opt/sdfs/volumes/VOLUMENAME
du -hc
Compare to df -h
df -h

In my example:
df -h
10G 5.4G 4.7G 54% /mnt/volume1
du -hc
3.1G total

So I’m using 5.4GB of space, but it’s only recorded as 3.1GB. Not a bad savings right there.

Next I’ll fill up the 10GB drive I created and see how much it really saves.

So I wanted a second volume shared for vmware using NFS. I ended up installing webmin and doing some of the configuration there for samba/cifs and nfs, but here’s my notes for commands on the opendedup server:

apt-get install nfs-kernel-server
nano /etc/exports
Add /export/volume2 *(insecure,no_subtree_check,rw,nohide,fsid=0)
/etc/init.d/nfs-kernel-server restart

Delete a Volume
If you ever need to delete a volume, make sure that you unmount it first:
umount /mnt/volume1
Then you can remove the data files:
rm -rf /opt/sdfs/volumes/volume1
The log files:
rm /var/log/sdfs/volume1*
And finally the configuration files:
rm /etc/sdfs/volume1*

Show Volume Info
/sbin/sdfscli --volume-info

I attempted to install OpenIndiana 151a on a Dell Poweredge 2650 server (2.8 xeon with 12gb ram, 5x 146SCSI, damn 3di controller). I put each disk into it’s own container on the controller and left the BIOS set to RAID.

I was receiving the following errors when attempting to install:

WARNING: Ignoring false simplex bit.
WARNING: Ignoring false simplex bit.
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: abort request, target=0 lun=0
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: abort device, target=0 lun=0
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: reset target, target=0 lun=0
WARNING: /pci@0,0/pci-ide@f,1/ide@0 (ata0):
timeout: reset bus, target=0 lun=0

IDE @ 0, the only IDE device I had was a CDROM drive. I figured it was a bad burn of the ISO – I verified the MD5 of the ISO I downloaded was correct, then I burned it again to CD (at 8X instead of 24X, just in case). Same issues persisted.

Looks like I need to disable DMA on the IDE channel to get this to work.
edit the grub configuration and add this at the end of the Kernel:
-B atapi-cd-dma-enabled=0,atapi-other-dma-enabled=0
Then run from that

I also ran the napp-it plugin:
wget -O - www.napp-it.org/amp | perl

Nexentastor Community edition did not want to run reliably on this hardware. I know it’s only 32-bit, so that is probably part of the issue. EON was just a giant pita for me, so I dropped that. FreeNAS was easy to setup but lacked on the ZFS support I wanted.

I don’t forget this, but I do have to teach people from time to time. Easiest way I know:
find / | grep FILENAMEYOUWANTTOFIND

so
find / | grep index.php

To find a file with certain words in the file:
grep -r ‘word or words you want to find’ /path/to/files/
grep -r ‘‘ /var/www/