We need the number of $3\times 3$ matrices whose entries are only $0$ or $1$.

A $3\times 3$ matrix has $9$ entries total. Each entry can be either $0$ or $1$.

The sum of all entries of such a matrix is simply the number of $1$'s in the matrix.

We want this sum to be a prime number. Since the number of $1$'s can range from $0$ to $9$, the possible prime sums are $2,3,5,7.$

For a fixed number $k$ of ones, the number of such matrices is the number of ways to choose which $k$ of the $9$ positions contain $1$: $\binom{9}{k}.$

Hence the required number is $\binom{9}{2}+\binom{9}{3}+\binom{9}{5}+\binom{9}{7}.$

Compute each term: $\binom{9}{2}=36,$ $\binom{9}{3}=84,$ $\binom{9}{5}=\binom{9}{4}=126,$ $\binom{9}{7}=\binom{9}{2}=36.$

Add them: $36+84+126+36=282.$

Therefore, the number of such matrices is $\boxed{282}.$

Comparison with stored correct answer: