Graphs are normally scaled to absolute dimensions of the user's perspective. So just add the absolute padding at the end, which fits the user.

If you can only add relative paddings before scaling, then calculate it by dividing the absolute one thru the scaling factor.