Random MomBlog

Since I know about Backup Bouncer, I have no excuse not to try it on rsync, the command-line way I use a USB flash drive to keep my files synchronized between work and home laptops. You can get started from this post. Still, I hadn’t tested rsync because my process worked as far as I could tell. (Then I ran into OpenMeta, and I want those tags that are in extended attributes!)

Here’s how I got started:

download Backup Bouncer
tar -zxf backup-bouncer-0.1.3.tgz
cd backup-bouncer-0.1.3
make
./bbouncer create-vol Src
./bbouncer create-vol Dst
./bbouncer create /Volumes/Src

So here are the results from the tests I ran.

First, rsync from an old version of RsyncX (now vanishware?) I believe:

/usr/local/bin/rsync --version
rsync  version 2.6.0  protocol version 27
Copyright (C) 1996-2004 by Andrew Tridgell and others
HFS+ filesystem support for OSX (C)2004 Kevin A. Boyd

Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
              IPv6, 32-bit system inums, 64-bit internal inums
	
sudo /usr/local/bin/rsync -av –eahfs –delete –showtogo –stats /Volumes/Src/ /Volumes/Dst/
	
sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying:    basic-permissions … ok (Critical)
Verifying:           timestamps … ok (Critical)
Verifying:             symlinks … ok (Critical)
Verifying:    symlink-ownership … FAIL
Verifying:            hardlinks … FAIL (Important)
Verifying:       resource-forks …
   Sub-test:             on files … ok (Critical)
   Sub-test:  on hardlinked files … FAIL (Important)
Verifying:         finder-flags … FAIL (Critical)
Verifying:         finder-locks … ok
Verifying:        creation-date … ok
Verifying:            bsd-flags … ok
Verifying:       extended-attrs …
   Sub-test:             on files … FAIL (Important)
   Sub-test:       on directories … FAIL (Important)
   Sub-test:          on symlinks … FAIL
Verifying: access-control-lists …
   Sub-test:             on files … FAIL (Important)
   Sub-test:              on dirs … FAIL (Important)
Verifying:                 fifo … ok
Verifying:              devices … FAIL
Verifying:          combo-tests …
   Sub-test:  xattrs + rsrc forks … FAIL
   Sub-test:     lots of metadata … FAIL

Yeah, I noticed that it fails on extended attributes *sigh*.

Next, rsync from MacPorts.

/opt/local/bin/rsync --version
rsync  version 3.0.5  protocol version 30
Copyright (C) 1996-2008 by Andrew Tridgell, Wayne Davison, and others.
Web site: http://rsync.samba.org/
Capabilities:
    64-bit files, 32-bit inums, 32-bit timestamps, 64-bit long ints,
    socketpairs, hardlinks, symlinks, IPv6, batchfiles, inplace,
    append, ACLs, xattrs, iconv, symtimes, file-flags
	
sudo ./bbouncer clean /Volumes/Dst
sudo /opt/local/bin/rsync -av --eahfs --delete --showtogo --stats /Volumes/Src/ /Volumes/Dst/
rsync error: syntax or usage error (code 1) at main.c(1423) [client=3.0.5]
sudo /opt/local/bin/rsync -av --xattrs --delete --stats /Volumes/Src/ /Volumes/Dst/
	
sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying:    basic-permissions ... ok (Critical)
Verifying:           timestamps ... ok (Critical)
Verifying:             symlinks ... ok (Critical)
Verifying:    symlink-ownership ... ok
Verifying:            hardlinks ... FAIL (Important)
Verifying:       resource-forks ...
   Sub-test:             on files ... ok (Critical)
   Sub-test:  on hardlinked files ... FAIL (Important)
Verifying:         finder-flags ... ok (Critical)
Verifying:         finder-locks ... FAIL
Verifying:        creation-date ... FAIL
Verifying:            bsd-flags ... ok
Verifying:       extended-attrs ...
   Sub-test:             on files ... ok (Important)
   Sub-test:       on directories ... ok (Important)
   Sub-test:          on symlinks ... ok
Verifying: access-control-lists ...
   Sub-test:             on files ... FAIL (Important)
   Sub-test:              on dirs ... FAIL (Important)
Verifying:                 fifo ... ok
Verifying:              devices ... ok
Verifying:          combo-tests ...
   Sub-test:  xattrs + rsrc forks ... ok
   Sub-test:     lots of metadata ... FAIL 

That looks a lot better! Now to compare to the rsync that Apple ships with Leopard (10.5).

/usr/bin/rsync --version
rsync  version 2.6.9  protocol version 29
Copyright (C) 1996-2006 by Andrew Tridgell, Wayne Davison, and others.

Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
              inplace, IPv6, 32-bit system inums, 64-bit internal inums
	
sudo ./bbouncer clean /Volumes/Dst
sudo /usr/bin/rsync -av –xattrs –delete –stats /Volumes/Src/ /Volumes/Dst/
rsync error: syntax or usage error (code 1) at /SourceCache/rsync/rsync-35.2/rsync/main.c(1333) [client=2.6.9]
sudo /usr/bin/rsync -av –delete –stats /Volumes/Src/ /Volumes/Dst/
	
sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying:    basic-permissions … ok (Critical)
Verifying:           timestamps … ok (Critical)
Verifying:             symlinks … ok (Critical)
Verifying:    symlink-ownership … ok
Verifying:            hardlinks … FAIL (Important)
Verifying:       resource-forks …
   Sub-test:             on files … FAIL (Critical)
   Sub-test:  on hardlinked files … FAIL (Important)
Verifying:         finder-flags … FAIL (Critical)
Verifying:         finder-locks … FAIL
Verifying:        creation-date … FAIL
Verifying:            bsd-flags … ok
Verifying:       extended-attrs …
   Sub-test:             on files … FAIL (Important)
   Sub-test:       on directories … FAIL (Important)
   Sub-test:          on symlinks … FAIL
Verifying: access-control-lists …
   Sub-test:             on files … FAIL (Important)
   Sub-test:              on dirs … FAIL (Important)
Verifying:                 fifo … ok
Verifying:              devices … ok
Verifying:          combo-tests …
   Sub-test:  xattrs + rsrc forks … FAIL
   Sub-test:     lots of metadata … FAIL 
	
sudo ./bbouncer clean /Volumes/Dst
sudo /usr/bin/rsync -av –extended-attributes –delete –stats /Volumes/Src/ /Volumes/Dst/
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-35.2/rsync/main.c(992) [sender=2.6.9]
	
sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying:    basic-permissions … ok (Critical)
Verifying:           timestamps … ok (Critical)
Verifying:             symlinks … ok (Critical)
Verifying:    symlink-ownership … ok
Verifying:            hardlinks … FAIL (Important)
Verifying:       resource-forks …
   Sub-test:             on files … ok (Critical)
   Sub-test:  on hardlinked files … FAIL (Important)
Verifying:         finder-flags … ok (Critical)
Verifying:         finder-locks … FAIL
Verifying:        creation-date … FAIL
Verifying:            bsd-flags … ok
Verifying:       extended-attrs …
   Sub-test:             on files … ok (Important)
   Sub-test:       on directories … ok (Important)
   Sub-test:          on symlinks … FAIL
Verifying: access-control-lists …
   Sub-test:             on files … ok (Important)
   Sub-test:              on dirs … ok (Important)
Verifying:                 fifo … ok
Verifying:              devices … ok
Verifying:          combo-tests …
   Sub-test:  xattrs + rsrc forks … ok
   Sub-test:     lots of metadata … ok 

Hmm, that took some wrangling, but it did work. Now to try the lart fork of rsync in MacPorts.

sudo port -f activate rsync-lart
--->  Activating rsync-lart
Warning: File /opt/local/bin/rsync already exists.  Moving to: /opt/local/bin/rsync.mp_1243889983.
Warning: File /opt/local/share/man/man1/rsync.1.gz already exists.  Moving to: /opt/local/share/man/man1/rsync.1.gz.mp_1243889983.
Warning: File /opt/local/share/man/man5/rsyncd.conf.5.gz already exists.  Moving to: /opt/local/share/man/man5/rsyncd.conf.5.gz.mp_1243889983.
	
/opt/local/bin/rsync --version
rsync  version 2.6.6  protocol version 29
Copyright (C) 1996-2005 by Andrew Tridgell and others

Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles,
              inplace, IPv6, 32-bit system inums, 64-bit internal inums
	
sudo ./bbouncer clean /Volumes/Dst
sudo /opt/local/bin/rsync -av –extended-attributes –delete –stats /Volumes/Src/ /Volumes/Dst/
rsync error: some files could not be transferred (code 23) at main.c(806)
sudo /opt/local/bin/rsync -av –extended-attributes –delete –stats /Volumes/Src/ /Volumes/Dst/
(now runs without error, but transfers stuff every time … known issue if you read lart link)
	
sudo ./bbouncer verify -d /Volumes/Src /Volumes/Dst
Verifying:    basic-permissions … ok (Critical)
Verifying:           timestamps … ok (Critical)
Verifying:             symlinks … ok (Critical)
Verifying:    symlink-ownership … ok
Verifying:            hardlinks … FAIL (Important)
Verifying:       resource-forks …
   Sub-test:             on files … ok (Critical)
   Sub-test:  on hardlinked files … FAIL (Important)
Verifying:         finder-flags … FAIL (Critical)
Verifying:         finder-locks … FAIL
Verifying:        creation-date … FAIL
Verifying:            bsd-flags … ok
Verifying:       extended-attrs …
   Sub-test:             on files … ok (Important)
   Sub-test:       on directories … ok (Important)
   Sub-test:          on symlinks … FAIL
Verifying: access-control-lists …
   Sub-test:             on files … ok (Important)
   Sub-test:              on dirs … ok (Important)
Verifying:                 fifo … ok
Verifying:              devices … ok
Verifying:          combo-tests …
   Sub-test:  xattrs + rsrc forks … ok
   Sub-test:     lots of metadata … FAIL

And then clean up.

eject Src and Dst disk images
rm Src.sparseimage Dst.sparseimage 

So, what did I learn?

They all fail on hardlinks (I don’t use ‘em, but Time Machine does), and all but Apple’s rsync fail on ACLs (but I don’t use ‘em).

The RsyncX rsync fails on symlink ownership (I use a little, but I can live with it), on finder flags (I use a little, but I hadn’t noticed a problem), on the extended attributes I now want (deal-breaker), and on lots of metadata (clearly I like metadata). It kicks out 12 failure messages, and I do need to upgrade so I can use OpenMeta.

Looking at rsync-lart, it fails on the finder flags (I use infrequently), finder locks (I can ignore), creation date (I like that to be correct), extended attributes on symlinks (I can avoid), and lots of metadata (but I love metadata), for 7 failure messages total. However, rsync-lart doesn’t have the performance of the others since it always transfers resource forks and extended attributes.

The latest version of rsync fails on finder locks (I don’t mind), creation date (that’s annoying, possibly too annoying), and lots of metadata (oops) for a mere 7 failure messages.

Apple’s rsync fails on finder locks, creation date, and extended attributes on symlinks (I can avoid that). Only 5 failure messages!

So Apple’s own rsync looks promising! When I throw out the FAIL messages that don’t bother me (hardlinks) and decide not to worry about creation dates initially, then all I’m left with is that it fails on the extended attributes of symlinks. I don’t care, since I intend to add extended attributes to the original files instead. I don’t care about ACLs, but I do care about lots of metadata where the rsync 3.0.5 failed.

That means the maligned Apple version of rsync looks ok to me. Now to see if it works out for me in practice …