A light-weight tar-like incremental backup program.

Abstract

can is a utility to create and manage incremental snapshots of parts of the filesystem for backup purposes. It is very similar to the tar program in terms of command line options and handling.

For details read the manpage.

Download

Current release 2.1 final:

• 2010-02-28:
  can-2.1.tar.gz (29.3 KB)
  
  
  • Move the subversion repository to git.
  • Fix directory name matching when using the -t/--list and -x/--extract actions.
  • Improve -t/--list output.
  • Use nanosecond accuracy for all timestamps.
  • When creating a snapshot, do not terminate upon unreadable files.
  • Fix logging of undecodable file names.
  • Exit with a non-null exit code on SIGINT.
  • Let the -t/--list command ignore corrupt snapshots.
  • Sort -t/--list output (alphabetically, size, reverse).
  • Arrange -t/--list output in columns.
  • Add -N as an alias for the --new option.
  • Group the help output.
  • Add xz/lzma2 compression (-y/--xz and -Y/--xz-all options).

Please direct bug reports and other requests to lars@gustaebel.de.

Source repository

The development source code for can is available from a public git repository. You can create your own clone of it:

    $ git clone git://gustaebel.de/lars/can.git

History

2.0 final - 2009-06-21: download

• Extended the compressed.list file.

• Fix the FileReader for files that have errors.
• Fix reading of COMPRESS_GZIP and COMPRESS_BZIP2 snapshots.
• Do not use st_dev and st_ino for file comparison as they are not always consistent between runs (e.g. on NFS).

• Added --exclude-tag option, including standard CACHEDIR.TAG support.
• Added -K/--keyfile option allowing blowfish encryption.
• -v/--verbose produces verbose output with -t/--list.

• Still find previous snapshots if they have been moved.
• Replace undisplayable characters in -t/--list output.
• Expand shell patterns in Actions.create() instead of FilesystemReader and fix some related defects.

• Fix decompression of snapshots.
• Create intermediate directories and do not fail on existing files.

• Fix a unicode conversion error in connection with symlinks.

• Fix a typo that causes errors with bzip2 compression.

• Added -Z and -J options that allow to compress the entire snapshot file instead of only compressable members.
• Do not store times before epoch.
• Added --version option.
• Removed last remains of segment size support.
• Fixed: Decode filenames read from --files-from option to unicode.

• removed use of tarfile module (pax format is too bloated), created a new format that allows very small snapshot files.

• Uses at least Python 2.5.
• Added a new archive format using only one archive for each snapshot instead of an index and a data file. Previous formats are no longer supported.
• Removed the -o/--output option.

• Added -M/--max-size option.
• Use Python 2.4.

• Fixed missing variable in filesystem.py.
• Fixed null regex in FilesystemReader so that it never matches.
• Fixed FileReader so that it sorts the input and removes duplicates by itself.
• Improved svn2cl.sh output with my own version of svn2cl.sh.
• Scrape version number for manpage out of libcan/__init__.py.
• Set base directory (-C option) to different defaults depending on action.
• Improved error handling and output.
• Exit on more fatal errors.

• Fixed: unable to match a whole directory if there is a trailing slash.

• Use man2html to generate a html version of the manpage.
• Updated setup.py for registering with PyPI.
• Use svn2cl instead of cvs2cl for autogenerating changelog.
• Catch KeyboardInterrupts more elegantly.
• Added needed import of errno module.

• Stop on ENOSPC error.
• Catch errors in the Differ iterator and display them.
• Raise an error in FilesystemReader if a given path is not inside the root.
• Added an EXAMPLE section that illustrates how to use can.
• Provide a default read_next_record() method.
• Introduced backward-compatibility. CAN1 format can still be read but will not be produced anymore.
• Compare each filename that is stored to the previous one and cut all leading characters that are equal.
• Changed struct format of File's data.
• Added -l/--one-file-system option.
• compress(): Seek away from the beginning to detect if a file is compressed. Data may not be representative.

• Fix: Make -C option behave more like expected.
• Fix: Sort globbed filenames before addition.

• Added Makefile that generates a CHANGES file (using cvs2cl.pl) and the manpage in the doc directory.
• Fixed wrong note about the default of -C/--directory.
• Fix: There was a nasty error in the FilesystemReader iterator. Non-directory files that were part of the command line (e.g. after globbing) were simply not added.
• Added manpage can.1 to the Manifest template.
• Improve FileReader for very large files with many filenames. Do not build a list with File objects at once, but construct it on-the-fly with a generator.
• Catch EnvironmentErrors on add().
• Added -T/--files-from option that makes it possible to read filenames to add from a file or stdin.

• Added MANIFEST.in to include COPYING in source distribution.
• Added GPL.
• compressed(): Do not decide whether a file is compressed on extension but try to compress a part of it and see if it is compressable. This is done only for files >= 512k in order not to slow the creation process.
• Switch off posix mode for tarwriter to be able to store long filenames.
• Regrouped -o/--output option.
• Install man page.
• Add -o/--output option which enables the user to convert a snapshot (or parts thereof) to a tar archive. It is written either to stdout or to a filename.
• Removed sorting (index-wise, position-wise, name-wise) from extract.
• Added seek() method to BZ2Reader.

• Fixed: Catch ArchiveError, that's raised when a file has a bad type.
• Put data files in a subdirectory named after the snapshot.
• Create snapshot only when files have been changed.
• Implemented improved extracting capabilities with segmented data files.
• Catch exception if snapshot exists.
• Added can manpage.
• Added segmentation (-S/--segment-size option) to data files.
• Added default index and compression attributes.
• File object now holds mode, mtime and size as attributes.
• Force compression method if it is already defined.
• Removed database.py which is no longer needed.
• Major rewrite: Comparison is done using sorted iterators. Design is much more elegant.
• If there are no new or modified files, only deleted files, force writing of the snapshot in an explicit manner.
• Snapshot handles conditional writing to disk itself now. Moved the code from DatabaseWriter to SnapshotWriter.
• Reworked SnapshotReader interface to not resemble a dict too much. Isolated Database.add().
• Raise ArchiveError if a file type cannot be archived.
• Enable find() to yield directories, too.
• Do not raise KeyError when name is not found.
• Check if the user passed a --file option for extract and list actions.
• Make Snapshots behave more like iterators wherever possible.
• Fixed compressed() function.
• Added some explanations to the zlib compresslevel = 0 performance hack.
• Improved logging, verbose output and output format.
• Removed superfluous cStringIO import.
• Removed --name option, the same can better be done using the --file option.
• Added PlainWriter that wraps a file object for writing.
• A more generic implementation of archive compression. It is now possible to choose between no compression, gzip and bzip2 compression using tar-like command line options.
• Ported the snapshot files from DBM files (which were very expensive in terms of disk space) to our own obscure bzip2'ed file format.

License

can is distributed under the GNU GPL license.

can - Copyright © 2010, Lars Gustäbel (lars@gustaebel.de)
All rights reserved.

This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA