Class IndexedStorageFile

java.lang.Object
com.hypixel.hytale.storage.IndexedStorageFile
All Implemented Interfaces:
Closeable, AutoCloseable
public class IndexedStorageFile extends Object implements Closeable
IndexedStorageFile binary file format designed to contain a limited number of data blobs of arbitrary length accessible via a fixed index. The IndexedStorageFile provides built in blob compression via Zstd. The Zstd compression level is modifiable via the setCompressionLevel(int) method.

File Structure 

Name         | bytes         | type              | notes
---------------------------------------------------------------------------------
Magic        | 20            | ASCII             | MAGIC_STRING
Version      | 4             | int               | VERSION
Blob Count   | 4             | int               | DEFAULT_BLOB_COUNT
Segment Size | 4             | int               | DEFAULT_SEGMENT_SIZE
Blob Indexes | BlobCount * 4 | int[Blob Count]   | UNASSIGNED_INDEX
Segments     | ...           | Segment[...]

Segment 

Name         | bytes | notes
-------------------------------------------------
Blob Data    | ...

Blob 

Name                 | bytes
-----------------------------
Blob Length          | 4
Compressed Length    | 4
Compressed Data      | ...

The IndexedStorageFile provides best effort safety and recovery if the application crashes at any point. This is achieved by ensuring the order of operations do not cause the file to ever be in an invalid state, however, if the underlying FileChannel manages to do a partial write for some specific values it is still possible for the file to become corrupted. Known Issue: There is two operations that currently break the above guarantee and is labeled with @Corruption. TODO: Future Work: It is possible to provide stronger guarantees of writes if we use some kind of signature validation for indexes.

The IndexedStorageFile is safe for use by multiple concurrent threads as long as only a single IndexedStorageFile object is used so that internal locking correctly operates. TODO: Future Work: This could be improved by sharing the locks between IndexedStorageFile objects based on the target file's Path. Could use reference counting along with a Cleaner.

Since:
2020/02/20
Version:
1.0