From the 3 points on the first plane you can determine its equation. Ditto, for the three mapped point on the second plane.

With the equations of both planes, you can determine the line where they intersect (assuming they are not parallel).

That line forms the axis of rotation; and the angle between the planes, the angle of rotation. You know have all the information required to build the matrix for the affine transformation that you can then apply to the rest of the points in the original plane to map them to their new positions.

Thank you for that. It may have given me the way forward. I have found a reference
http://inside.mines.edu/~gmurray/ArbitraryAxisRotation/ArbitraryAxisRotation.pdf
that gives the matrix for rotation about an arbitary axis.
I am hoping that your references and this will solve the problem.

You are assuming that points on the intersecting line do not move. This is a huge assumption that is almost always wrong.

This is a huge assumption that is almost always wrong.

Care to expand on that by showing us a practical example?

---

Examine what is said, not who speaks -- Silence betokens consent -- Love the truth but pardon error.

"Science is about questioning the status quo. Questioning authority".

In the absence of evidence, opinion is indistinguishable from prejudice.

RIP PCW

Do you want to keep the distances between points the same?

How do you know these three points?

With a rotation matrix you can obviously move points around. I'm not sure if this solves your problem. I think you have to supply more information on what exactly you're trying to do. Also there are many ways to handle rotation matrices, I normally look at them as a set of Euler angles but you can also use quaternions or delve into Lie algebras?!

There is a lot of SW around to handle these types of geometry problems. It's often not about the complexity of the calculations but more about understanding the particular geometry problem. I use SPICE for this. For example to project four points (corner points of a Field-of-View) from one plane onto another. This often involves a change of coordinate system as well.

Thank you for your comments.
The distances between the points will remain the same since the points are the corners of a 2d shape that
I want to move from one plane to another.
I know these points since I know where the original 2D object has to be positioned in space.
I suspect your are correct that 'understanding' the issue is the major hurdle.

Giving it some extra thought and reading again through the other responses I think your best bet is to follow BrowserUK's advice. Let me add that the angle between two planes is given by the angle between the normal vectors. See Lines and Planes for some basic examples. You can use the rotation matrices as described in the paper (the link you provided yourself) or look them up in Wikipedia or Mathworld.

It's a pity you don't describe what problem your working on. A physics problem? Computer graphics? etc.

The simplest solution is to realize that for any point in the first plane, there must exist a and b such that the point can be written as (x1 + a*(x2-x1) + b*(x3-x1), y1 + a*(y2-y1) + b*(y3-y1), z1 + a*(z2-z1) + b*(z3-z1)). The affine transformation that takes the first plane to the second will map that point to (s1 + a*(s2-s1) + b*(s3-s1), t1 + a*(t2-t1) + b*(t3-t1), u1 + a*(u2-u1) + b*(u3-u1)).

Now given any point in the first plane, all you have to do is solve for a and b, then you've got your answer.

(Please note that the direction that BrowserUk pointed you in usually gives the wrong answers, and can't even be tried if the planes are parallel.)

I am not sure where I should add this since there are a number of threads.
Therefore I have added this to the end.
I have now implemented the general approach suggested by BrowserUK.
I have checked the result by
1. making sure that the rotated points do lie on the second plane
2. checking the distances between corresponding pairs of points on both planes.
Both checks work out fine.
So until I read the additional comments about this question I thought that the problme was solved! (I did appreciate that there is a failure condition for parallel planes but this special case is easy to deal with.)
My next task is to look at some of the situations mentioned and see what happens to my current solution.

Your biggest sanity check should be that the 3 points you started with should wind up in the specified target location. If that is not true then you've merely found a mapping of one plane onto the other, but not the desired one.