Sidecar file explained
Sidecar files, also known as buddy files or connected files, are computer files that store data (often metadata) which is not supported by the format of a source file.
There may be one or more sidecar files for each source file. There may also be "metadata databases" where one database contains metadata for several source files.
In most cases the relationship between the source file and the sidecar file is based on the file name; sidecar files have the same base name as the source file, but with a different extension. The problem with this system is that most operating systems and file managers have no knowledge of these relationships, and might allow the user to rename or move one of the files thereby breaking the relationship.
Examples
- Amiga Hunk metadata
In AmigaOS, a file with a extension contains metadata for a companion Amiga Hunk executable file.
- Extensible Metadata Platform
Extensible Metadata Platform (XMP) metadata is stored in a sidecar file when either a file format does not support embedded XMP metadata or if the workflow requires this.
- DxO sidecar
Similar to the XMP format, DOP sidecars store metadata and edits made through DxO PhotoLab[1] and other DxO tools[2]
- Connected Web Files and Folders
A file system object that associated two or more files. The file system treats connected files as a unit for purposes of moving, copying, and deleting. Some versions of Internet Explorer and Microsoft Word can save an HTML and its hyperlinked assets as such a unit.
- THM
Many digital cameras will store a (thumbnail) file alongside a recorded movie, with the same base filename as the movie file. These thumbnail files are JFIF-encoded image files. This system allows for quickly displaying a still preview of the movie, and storing camera data which is not supported by the AVI file format.
- INF
Acorn filesystems support metadata such as load and execution addresses that may not be natively supported on other filesystems. A file is used to store this metadata in text format, stored in a file with the same base filename, e.g. Menu and, and .
- JPEG + WAV
Some digital cameras allow for voice/audio annotations with photos. These are then stored as WAV audio files alongside the JPEG photo file, with the same base filename.
- PDF + annotationsPDF viewers which allow the reader to annotate documents with comments and drawings may store these in a sidecar file, such as Xournal's files.
- RunPacker
The MS-DOS-based self-extracting archive generator RunPacker relied heavily on sidecar files because the proprietary package format PFA (packfile archive) used in it did not natively store file attributes or timestamps. To address this limitation, a backward-compatible system of auxiliary files (as called in official documentation) was implemented to store aforementioned data in volumes without changing their format. These were added to the PFA volume the same way as ordinary files, except they were marked by special characters in their name, and software made aware of this system (sometimes called ePFA, Extended PFA) processed them transparently. Older programs relying on the PFA format would simply read or extract the file along with all others found in the volume. The ePFA format was designed with further extensibility in mind, and has been actually extended with several features over time.
- Meta Information Encapsulation (MIE)
Meta Information Encapsulation sidecar files. The MIE format is an extensible, dedicated meta information format part of ExifTool. MIE files can be used to encapsulate meta information from many sources and bundle it together with any type of file.A variation of this are copies of the source file which contain largely the same information, but in a different format or from a previous version:
- Exif
Since many JPEG editing software used to destroy Exif metadata stored in digital photos, some photo cataloging applications can extract the Exif data and store that in an file, so that the metadata can later be re-inserted into the JPEG file.
- Raw + JPEG
Many digital cameras allow to store both uncompressed raw data and a JFIF-encoded image file when shooting in raw mode. This allows for faster previewing the photo, and support by applications that do not support the (often undocumented) raw format.
- TIF + TFW or JPG + JGW
Aerial photos may be supplied with a World File that determines the location, size and rotation of the image.Alternatives
Rather than storing data separately, it can be stored as part of the main file. This is particularly done for container files, which allow certain types of data to be stored in them. Instead of separate files on the file system, multiple files can be combined into an archive file, which keeps them together, but requires that software processes the archive file, rather than individual files. This is a generic solution, as archive files can contain arbitrary files from the file system.
Forks
See main article: Fork (file system). A file system level solution for the same problem are forks, which allow multiple pieces of data to be associated with a single file. Sidecar files can be seen as "forks for file systems without native support for forks".
These can then be manipulated with usual file system tools: because the support is built into the operating system, these resource forks will not show up as separate files, and all applications inherit support for resource forks. However, forks cannot be copied to file systems without support for forks, or transmitted over a channel that does not support forks. For interchange forks are generally instead stored as sidecar file.
The classic Mac OS and macOS are notable examples of operating systems with support for forks, in the HFS file system. However, this causes problems with exchanging over ISO 9660 format CD-ROM, FAT format MS-DOS disks, and over internet email, and requires the use of sidecar files to store this information. Microsoft NTFS supports Alternate Data Streams which are similar.
References
Notes and References
- https://www.dxo.com/fr/dxo-photolab/
- https://tutodxo.com/en/metadata-management-support/
