Given density distribution

$\rho(r)=\rho_0\left(1-\frac{r^2}{R^2}\right)$

We need the point inside the planet where gravitational field $g(r)$ is maximum.

Mass enclosed within radius $r$

For a spherically symmetric body,

$dm=\rho(x)\,4\pi x^2\,dx$

So,

$M(r)=\int_0^r 4\pi x^2\rho_0\left(1-\frac{x^2}{R^2}\right)dx$

$M(r)=4\pi\rho_0\int_0^r \left(x^2-\frac{x^4}{R^2}\right)dx$

$M(r)=4\pi\rho_0\left(\frac{r^3}{3}-\frac{r^5}{5R^2}\right)$

Gravitational field at radius $r$ inside the planet

By shell theorem,

$g(r)=\frac{GM(r)}{r^2}$

Hence,

$g(r)=\frac{G}{r^2}\cdot 4\pi\rho_0\left(\frac{r^3}{3}-\frac{r^5}{5R^2}\right)$

$g(r)=4\pi G\rho_0\left(\frac{r}{3}-\frac{r^3}{5R^2}\right)$

Condition for maximum field

Differentiate with respect to $r$:

$\frac{dg}{dr}=4\pi G\rho_0\left(\frac{1}{3}-\frac{3r^2}{5R^2}\right)$

For maximum,

$\frac{dg}{dr}=0$

$\frac{1}{3}-\frac{3r^2}{5R^2}=0$

$\frac{3r^2}{5R^2}=\frac{1}{3}$

$r^2=\frac{5R^2}{9}$

$r=\sqrt{\frac{5}{9}}\,R$

Check that it is maximum

$\frac{d^2g}{dr^2}=4\pi G\rho_0\left(-\frac{6r}{5R^2}\right)<0 \quad \text{for } r>0$

So this gives a maximum.

Option matching

$r=\sqrt{\frac{5}{9}}R$

This corresponds to Option D.

Comparison with stored answer

Stored correct answer: D

Derived answer: D

They agree.