Have you considered reading the xref table before beginning your manipulations, using it to calculate the sizes of the objects, reading the objects as binary records (set $/ to a reference to a number or use read) using the xref information, and then simply calculating and writing a new xref table? That should be faster still than asking the regex engine to scan the entire contents of a PDF.

therefor my regex forces this byte sequence to be found at the beginning of a line

A binary stream can also contain an end-of-line sequence, especially if we consider maliciously crafted input.