I managed to find flags of similar build of perl (just lower version) and checked out its flags with perl -V. I now built newer version of perl with the same flags and it seems it worked.

Thank you guys for all suggestions!