When the mode is GL_OBJECT_LINEAR, the texture coordinate is calculated using the the plane-equation coefficients that are passed via glTexGenfv(GL_S, GL_OBJECT_PLANE, planeCoefficients).
The coordinate (S in this case) is computed as:
S = Ax + By + Cz + D
using the [x y z] coordinates of the vertex (the values passed to glVertex).
If [A B C] is a unit vector, this means that the texture coordinate S is equal to the distance of the vertex from the texgen plane.