I Built a Python Compressor Using OpenAI's Tokenizer. Here's What Happened

LLMs spent billions of dollars learning the best subword dictionary that has ever existed. Somewhere around 200,000 pieces of language, ranked by how much they compress. And I kept looking at that dictionary and thinking: nobody is allowed to just use that for compression? nobody is allowed to just use that for compression? TokPress is a pure-Python lossless compressor that tokenizes input with OpenAI's o200k_base vocabulary, applies token-level LZ77, and entropy-codes the result with rANS. For many small, schema-homogeneous records (logs, JSON, telemetry) a trained dictionary takes the ratio to 0.2565, and a batch mode reaches 0.0875x — while compressing each record alone can inflate it past 1.0. It beats gzip on prose and loses honestly to zstd's trained dictionary.