Balrogg: Demonically compacting (up to 15%) lossless Vorbis/Opus recompressor

Posted by palaiologos 5 days ago

Comments

Comment by tux3 3 days ago

So, it looks like this tries to build a model (or rather, multiple layers of models combined with something called context mixing[0]), potentially analyzing the whole file first? Is it fair to call this a very fancy entropy coder, replacing the last phase of what a traditional audio codec does (the lossless part, after quantization)?

I like the code style, this is pretty nice C. It's refreshing to see human code.

[0]: https://en.wikipedia.org/wiki/Context_mixing

Comment by RicoElectrico 2 days ago

> I like the code style, this is pretty nice C.

Or is it? https://github.com/iczelia/balrogg/blob/7008bdc10ff25ad87105...

Comment by tux3 2 days ago

I admit I'm not a fan of the loop macros, or writing constants like 0xd80000 in decimal, or having all of that code inlined with no further explanation or git history.

But I think it's just that this file is lifting parts of the logic from PAQ8 (the inlined num_states/next_state named in a comment sure look exactly like PAQ functions translated into C, which, kudos to PAQ for having extensive comments in their code by the way).

I'm a bit disappointed that PAQ isn't mentioned in the README, and only very obliquely in the code, if that's where those functions come from. PAQ is GPLv3, at least a mention could have been nice (assuming I'm not wrong about the source of that logic).

Comment by palaiologos 2 days ago

I use the PAQ terminology as it is likely to be familiar to other compression experts. However, the lineage to PAQ is very, very limited, and many other compressors also re-use its common ideas.

Comment by tux3 2 days ago

I see, my bad. I tried to follow the comment and jumped to conclusions, sorry about that.

Comment by genxy 2 days ago

This code looks like Fortran55

Comment by merbanan 3 days ago

Vorbis sends the entropy tables in the actual file at the start.

https://www.xiph.org/vorbis/doc/Vorbis_I_spec.html#x1-470003

If you have the complete file you can calculate the optimal ones. In a streaming scenario that does not work and you have use something you think is good.

Comment by eru 1 day ago

I wonder if you could send new tables every so often in your stream?

Comment by sinnet3000 3 days ago

The git username sounded familiar, it's from the same developer that created bzip3.

Comment by slyall 2 days ago

Yes.

https://github.com/iczelia/bzip3

Comment by BoingBoomTschak 3 days ago

Those .blr extensions in the examples are making me doubt: the result is still conformant and playable ogg Vorbis/Opus, right?

Comment by sinnet3000 3 days ago

From what I can see, it's more like a specialized ZIP-like compressor for Vorbis/Opus files. You archive them on blr, and decompress them back to the original format.

Comment by ncruces 3 days ago

Kinda like what Dropbox Lepton did for JPEG back then.

https://github.com/dropbox/lepton

Comment by BoingBoomTschak 2 days ago

Ah, it's basically a PAQ context.

Comment by embedding-shape 2 days ago

Simpler: it's basically ZIP but specific to one format.

Comment by bigC5560 3 days ago

I just tested this out for myself and I was unable to play the .blr file in VLC or using ffplay. It is not recognized as anything by ffprobe either. So it seems like you need to decompress it to play it in a standard player unfortunately.

Comment by RicoElectrico 2 days ago

There is a long history of compressors that replace the entropy coding layer of a codec with something else entirely. This is exactly it.

Comment by EdSchouten 3 days ago

Interesting that it doesn’t use the same open source license as Vorbis/Opus itself. Otherwise improvements like these could be upstreamed?

Comment by functionmouse 2 days ago

Why was this written with explicit Windows 95 support? Just 'cuz?

Comment by lightedman 2 days ago

Probably the same reason it was written with explicit DOS support.