We need to count the number of $2\times 2$ matrices

where each entry is from $\{0,1,2,3,4,5\}$ and

for some prime $p$ with $2<p<8$.

1. Possible values of the prime sum

The primes satisfying $2<p<8$ are

So we need the number of ordered quadruples $(a,b,c,d)$ with each entry in $\{0,1,2,3,4,5\}$ such that the sum is $3$, $5$, or $7$.

Thus required number is

where $N_s$ is the number of solutions of

with $0\le a,b,c,d\le 5$.

2. Count solutions for sum $3$

We need nonnegative integer solutions of

Since $3<6$, the upper bound $\le 5$ is automatically satisfied.

By stars and bars,

3. Count solutions for sum $5$

We need nonnegative integer solutions of

Again, since $5\le 5$, the upper bound is automatically satisfied.

So,

4. Count solutions for sum $7$

We need solutions of

with $0\le a,b,c,d\le 5$.

First ignore the upper bound. Total nonnegative solutions:

Now subtract solutions where some variable is at least $6$.

Suppose $a\ge 6$. Let

Then

Number of such solutions is

Similarly for $b,c,d$, so total bad solutions:

There cannot be two variables simultaneously $\ge 6$, because then the sum would be at least $12>7$.

Hence,

5. Total number of matrices

Therefore,

So the number of such matrices is

6. Comparison with stored answer

Stored correct answer: $180$

Our derived answer is also $180$, so they agree.