HELIOS “mkisofs” – the ultimate UDF disk image backup utility

The free public domain disk image utility “mkisofs” received major enhancements by HELIOS Software engineers, to enable flexible disk image creation for archiving and backup tasks going way beyond DVD image creation. HELIOS added support to archive hundreds of thousands of files/folders into disk images without any size limitations.


Helmut Tschemernjak, president and founder of HELIOS Software notes: “Today's growing disk capacities and inexpensive external disk systems allow storing read-only archives quickly on disk instead of burning media or using proprietary tape formats. We turned “mkisofs” into a general purpose solution which allows archiving Mac, Windows, and UNIX specific files with their attributes into UDF disk images which can easily be mounted by Mac and Windows users.”.

Download HELIOS “mkisofs”

When to use which utility for what purpose?

For backups it may be vitally important to store the data read-only because this will prevent the data from being (unintentionally) manipulated. The table below lists some backup utilities and shows whether the resulting backup can be read (mounted) on other platforms, and whether it can be created as a read-only archive. This illustrates important differences between creating a UDF (Universal Disk Format) disk image and simply copying or cloning to a backup drive or a Mac “Disk Utility” disk image.

Data is accessible on the following platforms

Solution

Mac

Win

UNIX

Read-only

HELIOS mkisofs

+

+

 +*

+

Time Machine (Apple)

+

Carbon Copy Cloner (Bombich Software)

+

HELIOS Xtar

n/a

rsync (Andrew Tridgell)

+

+

 +*

n/a

Disk Utility (Apple)

+

+

* Meta data such as Mac resources, Finder info or Windows attributes get lost

mkisofs GUI tool by UGSoft

UGSoft developed a useful utility to operate HELIOS “mkisofs” from a GUI. This comes in handy for those who are not too familiar with CLI programs. The “mkisofs Tool” is free of charge and can be downloaded from the UGSoft website.


Screenshot of the “mkisofs Tool” frontend

Disk image format background information

“mkisofs” supports the creation of ISO-9660, Rock Ridge extensions, Mac HFS, Windows Joliet and UDF disk images whereas only the Rock Ridge and UDF formats will, with the new HELIOS enhancements, support large amounts of files/folders. Only the UDF format can handle disk images cross-platform on a Mac and Windows system.

For mounting backup disk images on UNIX

Rock Ridge is the perfect format because it handles all special UNIX attributes.

For mounting backup disk images from Mac or Windows

UDF is the perfect format because it preserves all important Mac and Windows file attributes and can easily be mounted on a Mac or Windows Vista by clicking on the image file, or on Windows XP by using the free (unsupported) Microsoft Virtual CD-ROM Control Panel, which maps the disk image to a new drive letter. An alternative is DAEMON Tools.

UDF format capability overview

UDF Feature

Mac

Win

Comment

Long file names

+

+

Large files (> 4 GB)

+

+

Support added by HELIOS

Unicode file names

+

+

Support added by HELIOS

Windows attributes

+

+

Support added by HELIOS

UNIX permissions

+

Not used

Support added by HELIOS

UNIX symbolic links

+

Support added by HELIOS

Large UDF images (> 500 GB tested)

+

+

Support added by HELIOS

Millions of files / folders

+

+

Support added by HELIOS

Mac Type / Creator

+

Not used

Support added by HELIOS

Mac resource file

+

Not used

Support added by HELIOS

Not seen by Windows


UDF drawbacks

Symbolic links are turned off by default because Windows will issue error messages when copying / accessing symbolic link files. Current Windows XP / Vista versions will not handle UDF Windows file streams, therefore additional file stream data is not available on Windows. The file creation date is not supported, instead the last modification date is used.

UDF generated images have been tested on Windows XP-SP2, Windows Vista, Mac OS X 10.3 or newer (PPC & Intel).

Rock Ridge format capability overview

The Rock Ridge format can handle all UNIX file attributes including long file names and their attached file permissions. The Rock Ridge format can be mounted on a UNIX server and re-published via AFP using HELIOS EtherShare, CIFS using HELIOS PCShare, and WebShare to remote Mac, Windows and Web users.

Rock Ridge drawbacks

  • Rock Ridge images cannot be mounted on Windows
  • Rock Ridge images can be mounted on a Mac but:
    • Mac specific information like resource file and type / creator information gets lost
  • Rock Ridge does not support input files larger than 4 GB (a future version of “mkisofs” should support Rock Ridge files larger than 4 GB)

The following data volumes are recommended (tested)

Volumes

Image format

Client compatibility

UNIX RAW

Rock Ridge via -R option

UNIX

UNIX HELIOS UTF-8

Rock Ridge via -R option

UNIX or re-published as AFP or SMB/CIFS via EtherShare / PCShare file server

UNIX HELIOS UTF-8

UDF via
-udf --heliosutf8 -no-hfs
option

Disk image can be mounted by Mac and Windows (directly or from a HELIOS volume)

Mac OS X HFS

UDF via
-udf --osx-hfs -no-hfs
option

Disk image can be mounted by Mac and Windows (directly or from a HELIOS volume)


The following archive configurations have been tested (completely verified)

  • Backup of more than 150.000 folders
  • Backup of more than 300.000 files
  • Backup of very large single files (> 30 GB)
  • UDF disk images larger than 500 GB
  • Backup of modified / missing files after backup started (using the -ignore-changed-files option)


The following “mkisofs” options are important

Option

Comment

-nolimit-pathtables

Added by HELIOS; will not limit the directory depth

-ignore-changed-files

Added by HELIOS; will continue with archive even if files are changing (warning will occur, changed files will contain zeros)

-no-hfs

Will omit an HFS disk image creation; the “mkisofs” built-in HFS image creation functionality cannot handle many / large files

-create-udfsymlinks

Added by HELIOS; will preserve symlinks on UDF. It is turned off by default because symlinks will produce error messages on Windows

-iso-level 4

Is always required, and allows deep trees to be saved. It is supported by Mac / Win / UNIX.

For space reasons, we replaced in the following examples the default options string with $options:
$options = "-iso-level 4 -nolimit-pathtables -ignore-changed-files -no-hfs"

“mkisofs” HELIOS UTF8 volume UDF backup directory example (complete)
Use the following command to backup the “/data” directory tree into a “data.img” image file:
mkisofs -udf $options --heliosutf8 -o /backup/data.img /data

“mkisofs” HELIOS UTF8 volume UDF backup directory example (changes since three days ago):
Use the following command to backup only files changed within the last three days from the “/data” directory tree into a “data.img” image file:
mkisofs -udf $options --heliosutf8 -o /backup/data.img -find /data -time 3

“mkisofs” UNIX RAW volume Rock Ridge backup directory example (complete):
Use the following command to backup the “/data” directory tree into a “data.img” image file:
mkisofs -R $options -o /backup/data.img /data

“mkisofs” Mac OS X HFS volume UDF backup directory example (complete):
Use the following command to backup the “/data” directory tree into a “data.img” image file:
mkisofs -udf $options --osx-hfs -o /backup/data.img /data

“mkisofs” Mac OS X HFS volume UDF backup directory example (changes since three days ago):
Use the following command to backup only files changed within the last three days of the “/data” directory tree into a “data.img” image file:
mkisofs -udf $options --osx-hfs -o /backup/data.img -find /data -time 3

HELIOS “mkisofs” Script Server sample script

HELIOS provides the Script Server sample script “archiver.pl”. This script allows creating a disk image when files and folders are copied into a hot folder. Since copying the data into the hot folder can take some time, depending on the amount of data, the disk image creation should only be started when the copying process has finished. This is done by renaming the folder with a specified suffix, which will start the script.

Prior to running the script, some environment variables must be adjusted according to your needs:

Environment variable

Example value

Description

burnFailedFolderName

ERROR

Folder to which failed data is moved

burnFinishedFolderName

DONE

Folder to which the successfully burned disk image is moved

burnInProgressFolderName

WORK

Temporary folder where the source and target data is stored while being processed. This folder is automatically removed after the script has finished

removeSourceData

false

If set to “true”, the source data is deleted after the script has finished

startBurningSuffix

.burn

Folder suffix which will start the script to run

The “archive.pl” script can be downloaded via HELIOS WebShare at:

Server:

http://webshare.helios.de

User name:

tools

Password:

tools

Login, go to the sharepoint: “HELIOS Tools” > “HELIOS Script Server” > “HELIOS Sample Scripts”, check “archiveNew.pl” and select “Download” from the “Transfer” menu.

Memory requirements (100.000 files require ca. 200 MB)
When creating disk images, “mkisofs” does this in a three-pass process: 

  1. Collect all files by reading the complete directory tree with the required attribute information into the memory
  2. Write the complete directory information (e.g. UDF) into the image file
  3. Append all file data content to the image file

Depending on the amount of files, directory depth and file name length, the passes 1 and 2 require about 2 kB memory per file / folder. This means a volume backup of 100.000 files requires about 200 MB of memory for the “mkisofs” process.

Note: UDF disk images are not compressed. The final UDF disk image size is about the total of all backed up files.

Background of “mkisofs”

“mkisofs” was mainly developed by the pioneer public domain engineer Jörg Schilling. Earlier versions are distributed with Sun Solaris, IBM AIX, Linux as well as part of many other solutions. “mkisofs” is now distributed as part of “cdrtools”. The source code is available at:

http://cdrecord.berlios.de/old/private/cdrecord.html.

HELIOS contributions to “mkisofs” have been submitted to Jörg Schilling for incorporation into the “mkisofs” source.

Download HELIOS “mkisofs” tested binary versions

Available from HELIOS: Mac OS X, Solaris, Linux, AIX. The currently supported HELIOS platforms can be viewed here »

All HELIOS enhancements are integrated into the latest version. The binary can be downloaded from:

Server:

http://webshare.helios.de

User name:

tools

Password:

tools

Login, go to the sharepoint: “HELIOS Tools” > “HELIOS mkisofs”, “<your platform>”, select mkisofs” and select “Download” from the “Transfer” menu.
At present there is no HELIOS version of “mkisofs” available for Windows. However, “mkisofs” will also run on Windows machines when compiled with Cygnus' “cygwin” tool (available from http://www.cygwin.com/mirrors.html) but it has not received any testing / development for the Windows platform by HELIOS.

Support disclaimer

Please note that “mkisofs” is a license-free utility (GNU or CDDL). HELIOS Software GmbH does not provide any support for this tool. Neither takes HELIOS responsibility for loss of data, etc.

Professional backup turnkey solutions

“mkisofs” is a great tool for archiving data and creating custom UDF disk images. However, professional backup solutions offer many more features and better support. HELIOS recommends the following solutions for backup, synchronization and archiving. Both solutions offer professional support options: