PDFs are really complicated, but also have significant value – precisely because they are so ubiquitous. PDFs are
the format for exchanging documents that shouldn't be edited (like invoices),
the format for distributing layouted content like print-ready files,
and with PDF/A the format for long-term electronic archiving.
While Adobe has tons of extensions and special features in some PDF variants, those are rarely used. The worst PDFs I had to deal with used embedded macros to dynamically generated additional pages and QR codes based on form input, or embedded multimedia into the page. That is not normal. Many people don't even use Adobe software for viewing PDFs, so anything more fancy than basic forms is not going to work reliably.
There is an open source ecosystem for dealing with PDFs. Tools like PdfTeX and Cairo can be used for PDF creation, Poppler for PDF parsing and rendering, Ghostscript for rasterization, iText for creation and manipulation. I'm not sure if LibreOffice has a custom PDF backend.
But, many of these use copyleft licensing (AGPL or GPL). Some of these use a dual-licensing scheme, where users can pay for a proprietary license if they don't want copyleft licensing terms. And commercial users do pay, because a good PDF API is so valuable.
The incentives of all involved actors are (mis-)aligned such that a permissively licensed Open Source PDF library ecosystem is unlikely to spring up:
- PDF is a complex format, so that there's a high barrier to entry in this “market”. Development costs for a “good enough” PDF library are high. The cost of the specifications is only a small aspect, compared to the time investment.
- Commercially motivated Open Source authors will have to run the numbers: is it cheaper to pay for development of a new PDF library, or is it cheaper to pay for a proprietary license for an existing library? Hint: PDF API vendors like iText know this and can price accordingly. This problem could be avoided by a large consortium (effectively crowd-funding creation of a new API), but so far no one has organized this.
- Philosophically motivated Open Source authors are probably entirely happy with copyleft-licensed libraries. While there is also a community of people that strongly prefer permissive Open Source licenses over copyleft licenses (greetings to the OpenBSD crowd), PDF manipulation is neither an integral part of computing nor that terribly exciting to motivate some individuals to start and complete this kind of project.