After playing around with ZFS for a while (it’s pretty nice, especially snapshots), I wanted to try non-solaris-type-deduplication. Opendedup was the answer.<\/p>\n
I installed on a 2.8GHz Xeon Dell Poweredge 850 with 2GB RAM, and a mirrored array 500GB SATA drives.<\/p>\n
Installed 10.04.3 LTS x64<\/p>\n
Upgrade the Server<\/strong> Install and Download<\/strong> Edit<\/strong> soft\tnofile\t65535 Create<\/strong> Mount<\/strong> Verify<\/strong> sdfs:\/etc\/sdfs\/volume1-volume-cfg.xml:6442 I then copied about 2GB worth of data. Looks like it parses through every 10 minutes or so – but since it’s the end of the day I’ll be checking in the morning.<\/p>\n There are a few common errors with simple fixes. The files are actually stored in \/opt\/sdfs\/volumes\/VOLUMENAME In my example: So I’m using 5.4GB of space, but it’s only recorded as 3.1GB. Not a bad savings right there.<\/p>\n Next I’ll fill up the 10GB drive I created and see how much it really saves.<\/p>\n So I wanted a second volume shared for vmware using NFS. I ended up installing webmin and doing some of the configuration there for samba\/cifs and nfs, but here’s my notes for commands on the opendedup server:<\/p>\n Delete a Volume<\/strong> Show Volume Info<\/strong> After playing around with ZFS for a while (it’s pretty nice, especially snapshots), I wanted to try non-solaris-type-deduplication. Opendedup was the answer. I installed on a 2.8GHz Xeon Dell Poweredge 850 with 2GB RAM, and a mirrored array 500GB SATA drives. Installed 10.04.3 LTS x64 Upgrade the Server apt-get dist-upgrade reboot Install and Download apt-get … Continue reading Opendedup Ubuntu<\/span>
\napt-get dist-upgrade<\/code>
\nreboot<\/p>\n
\napt-get install openjdk-6-jre<\/code>
\napt-get install attr<\/code>
\nwget http:\/\/opendedup.googlecode.com\/files\/sdfs-1.1.0_amd64.deb<\/code>
\nwget http:\/\/opendedup.googlecode.com\/files\/sdfs_fuse_2.8.4.tar.gz<\/code>
\ntar zxvf sdfs_fuse_2.8.4.tar.gz<\/code>
\ncd sdfs_fuse_2.8.4<\/code>
\ndpkg --install *.deb<\/code>
\napt-get install -f<\/code>
\ndpkg -i sdfs-1.1.0_amd64.deb<\/code>
\napt-get install -f<\/code><\/p>\n
\nnano \/etc\/security\/limits.conf<\/code>
\nAdd the following above # End of file:<\/p>\n
\nhard\tnofile\t65535<\/p><\/blockquote>\n
\n\/sbin\/mkfs.sdfs --volume-name=VOLUMENAME --volume-capacity=CAPACITY<\/code>
\nSo in my case I wanted a 10GB volume for testing:
\n\/sbin\/mkfs.sdfs --volume-name=volume1 --volume-capacity=10GB<\/code><\/p>\n
\nmkdir \/mnt\/volume1<\/code>
\n\/sbin\/mount.sdfs -v VOLUMENAME -m MOUNTPOINT<\/code>
\n\/sbin\/mount.sdfs -v volume1 -m \/mnt\/volume1<\/code><\/p>\n
\ndf -h<\/code><\/p>\n
\n 10G 0 10G 0% \/mnt\/volume1<\/p><\/blockquote>\n
\n1. OutOfMemoryError – This is caused by the size of the DedupStorageEngine memory requirements being
\nlarger than the heap size allocated for the JVM. To fix this edit the mount.sdfs script and increase the
\n-Xmx2g to something larger (e.g. -Xmx4g).<\/p>\n
\ndu -hc<\/code>
\nCompare to df -h
\ndf -h<\/code><\/p>\n
\ndf -h
\n 10G 5.4G 4.7G 54% \/mnt\/volume1
\ndu -hc
\n3.1G total<\/p>\napt-get install nfs-kernel-server<\/code>
\nnano \/etc\/exports<\/code>
\nAdd \/export\/volume2 *(insecure,no_subtree_check,rw,nohide,fsid=0)
\n\/etc\/init.d\/nfs-kernel-server restart<\/code><\/p>\n
\nIf you ever need to delete a volume, make sure that you unmount it first:
\numount \/mnt\/volume1<\/code>
\nThen you can remove the data files:
\nrm -rf \/opt\/sdfs\/volumes\/volume1<\/code>
\nThe log files:
\nrm \/var\/log\/sdfs\/volume1*<\/code>
\nAnd finally the configuration files:
\nrm \/etc\/sdfs\/volume1*<\/code><\/p>\n
\n\/sbin\/sdfscli --volume-info<\/code><\/p>\n","protected":false},"excerpt":{"rendered":"