The performance of interFoam (a widely used solver within OpenFOAM®) in simulating the propagation of a nonlinear (stream function solution) regular wave is investigated in this work. It is demonstrated that over time there is a tendency for surface elevations to increase, wiggles to appear in the free surface, and crest velocities to become (severely) overestimated. It is shown that increasing the temporal and spatial resolution can mitigate these undesirable effects, but that a relatively small Courant number is required and fine discretization is needed, indicating that many past simulations have not converged. It is further demonstrated that discretization schemes and solver settings (often treated as a "black box" by users) can have a major impact on the results. This impact is documented, and it is shown that obtaining a "diffusive balance" is crucial to accurately propagate a surface wave over long distances without requiring exceedingly high temporal and spatial resolutions. Finally, the new code isoAdvector is demonstrated to produce comparably accurate results to interFoam, while maintaining a sharper surface. It is hoped that the present study will raise awareness of potential shortcomings of interFoam and enable a more accurate and optimal use, in CFD simulation of free surface waves.