Overview

The first compression experiment explored whether long runs of repeated bytes inside .gguf files could be replaced with compact dictionary references.

The hypothesis was that GGUF metadata and tokenizer sections contain enough repeated values to produce meaningful lossless compression.

The experiment focuses entirely on repeated-byte sequences and does not modify tensor values.

Motivation

Early inspection of GGUF files showed many regions containing repeated bytes, especially inside headers, metadata, and tokenizer information.

Example:

00 00 00 00 00 00 00

Instead of storing every repeated byte individually, these sequences were replaced with marker-based references inside a custom binary format.

Compression Pipeline

GGUF File
    ↓
Read Raw Bytes
    ↓
Detect Repeated Runs
    ↓
Replace With Dictionary Markers
    ↓
Write BFG1 Format

Results

Observations

The approach performed well while processing metadata-heavy sections of GGUF files.

Compression effectiveness decreased rapidly as larger portions of quantized tensor data entered the sample.