\[\mathbf{M}\]
such that \[\mathbf{M}^n = \mathbf{0}\]
for some n.
A nilpotent matrix must be a square matrix, else could not find \[\mathbf{M}^n = \mathbf{0}\]
and it must have zero determinant, sine \[det (\mathbf{M}^n) =(det(\mathbf{M}))^n =0\]
Example:
\[ \left( \begin{array}{ccc} 0 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array} \right) \]
is nilpotent since
\[ \left( \begin{array}{ccc} 0 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array} \right) \left( \begin{array}{ccc} 0 & 0 & 1 \\ 0 & 0 & 1 \\ 0 & 0 & 0 \end{array} \right) = \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 0 & 0 \\ 0 & 0 & 0 \end{array} \right)\]
but
\[ \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array} \right) \]
is not nilpotent since
\[ \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array} \right) \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array} \right) = \left( \begin{array}{ccc} 0 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array} \right)\]
A nilpotent matrix must have zero determinant, but it does not follow that a matrix with zero determinant is nilpotent. The matrix above is a counterexample.