Yeah the nomenclature sucks.
When talking about the file, the two terms are often used interchangeably (and are usually a .diff or a .patch extension).
For fun, the GNU Patch manpage says:
"NAME: patch - apply a diff file to an original"
followed by:
"patch takes a patch file patchfile containing ..."
"patch tries to skip any leading garbage, apply the diff, and then ..."