If your shoreline data already knows which point is next down the beach, this becomes a problem in graph topology - grouping connected pieces of the shoreline "network". Pick a point and walk the list till you see the same point again. Pick an unvisited point, either an island or a neighboring lake, and repeat. This algorithm is linear in the number of points.

Chapter 8 of the wolf book covers this, but is geared to more complex graphs.

After Compline,
Zaxo