Let the A.P. have first term $a=3$ and common difference $d$.

Sum of first four terms: $S_4=\frac{4}{2}\left[2a+(4-1)d\right]=2(2a+3d).$ Since $a=3$, $S_4=2(6+3d)=12+6d.$

The next four terms are the $5^{\text{th}}$ to $8^{\text{th}}$ terms. Their sum is: $S_8-S_4.$ Now, $S_8=\frac{8}{2}\left[2a+(8-1)d\right]=4(2a+7d).$ With $a=3$, $S_8=4(6+7d)=24+28d.$ Hence, $\text{sum of next four terms}=S_8-S_4=(24+28d)-(12+6d)=12+22d.$

Given that the sum of the first four terms is equal to one-fifth of the sum of the next four terms: $12+6d=\frac{1}{5}(12+22d).$ Multiply by $5$: $60+30d=12+22d.$ $48=-8d$ $d=-6.$

Now find the sum of the first $20$ terms: $S_{20}=\frac{20}{2}\left[2a+(20-1)d\right].$ Substitute $a=3$, $d=-6$: $S_{20}=10\left[6+19(-6)\right]$ $=10(6-114)$ $=10(-108)$ $=-1080.$

Therefore, the correct option is: $\boxed{\text{C }(-1080)}$

Comparison with stored answer: Stored correct answer is C, which matches our result.