The life cycle of the scientific data is well defined: data is collected, then processed,
archived and finally deleted. Data is never modified. The original data is used or new,
derived data is produced: Write Once Read Many times (WORM). With this model in
mind, dCache was designed to handle immutable files as efficiently as possible. Currently,
data replication, HSM connectivity and data-server independent operations are only
possible due to the immutable nature of the stored data.
dCache is seen increasingly as a general-purpose file system, helped by its support for
the NFSv4.1standard,especially by new communities, such as photon science and
microbiology. Although many users are aware of the immutability of data stored in
dCache, some applications and use cases still require in-place update of stored files.
Satisfying these requires some fundamental changes to dCache's core design. However,
those changes must not compromise any aspect of existing functionality.
In this presentation we will show the new developments in dCache that will turn it
into a regular file system. We will discuss the challenges of building a POSIX-compliant
distributed storage system, one that can handle multiple replicas and that remains
backward compatible by providing both WORM and non-WORM capabilities within
the same system.