1. Given differential equation

   $\frac{dy}{dx}+\frac{xy}{x^2-1}=\frac{x^4+2x}{\sqrt{1-x^2}}, \qquad -1<x<1$

   with initial condition

   $f(0)=0.$

2. Rewrite in linear form

   Since

   $\frac{x}{x^2-1}=-\frac{x}{1-x^2},$

   the equation becomes

   $\frac{dy}{dx}-\frac{x}{1-x^2}y=\frac{x^4+2x}{\sqrt{1-x^2}}.$

   This is a linear differential equation:

   $\frac{dy}{dx}+P(x)y=Q(x),$

   where

   $P(x)=-\frac{x}{1-x^2}.$

3. Find the integrating factor

   $\text{I.F.}=e^{\int P(x)\,dx}=e^{\int -\frac{x}{1-x^2}\,dx}.$

   Let $u=1-x^2$, then $du=-2x\,dx$, so

   $\int -\frac{x}{1-x^2}\,dx=\frac12\int \frac{du}{u}=\frac12\ln(1-x^2).$

   Hence

   $\text{I.F.}=e^{\frac12\ln(1-x^2)}=\sqrt{1-x^2}.$

4. Multiply the equation by the integrating factor

   $\sqrt{1-x^2}\frac{dy}{dx}-\frac{x}{\sqrt{1-x^2}}y=x^4+2x.$

   The left side is

   $\frac{d}{dx}\big(y\sqrt{1-x^2}\big).$

   Therefore,

   $\frac{d}{dx}\big(y\sqrt{1-x^2}\big)=x^4+2x.$

5. Integrate

   $y\sqrt{1-x^2}=\int (x^4+2x)\,dx=\frac{x^5}{5}+x^2+C.$

   So

   $y=f(x)=\frac{\frac{x^5}{5}+x^2+C}{\sqrt{1-x^2}}.$

6. Use the initial condition

   Since $f(0)=0$,

   $0=\frac{0+0+C}{1} \implies C=0.$

   Thus

   $f(x)=\frac{\frac{x^5}{5}+x^2}{\sqrt{1-x^2}}.$

7. Set up the required integral

   We need

   =\int_{-\sqrt3/2}^{\sqrt3/2} \frac{\frac{x^5}{5}+x^2}{\sqrt{1-x^2}}\,dx.$$ Split into two parts: $$I=\frac15\int_{-a}^{a}\frac{x^5}{\sqrt{1-x^2}}\,dx+\int_{-a}^{a}\frac{x^2}{\sqrt{1-x^2}}\,dx, \qquad a=\frac{\sqrt3}{2}.$$

8. Use symmetry

   • $\dfrac{x^5}{\sqrt{1-x^2}}$ is an odd function, so its integral from $-a$ to $a$ is $0$.
   • $\dfrac{x^2}{\sqrt{1-x^2}}$ is even.

   Therefore,

   $I=\int_{-a}^{a}\frac{x^2}{\sqrt{1-x^2}}\,dx=2\int_0^a \frac{x^2}{\sqrt{1-x^2}}\,dx.$

9. Evaluate the integral

   Put $x=\sin\theta$, so $dx=\cos\theta\,d\theta$ and $\sqrt{1-x^2}=\cos\theta$ for $\theta\in[0,\pi/3]$ because $a=\sqrt3/2=\sin(\pi/3)$.

   Then

   $I=2\int_0^{\pi/3} \sin^2\theta\,d\theta.$

   Using

   $\sin^2\theta=\frac{1-\cos2\theta}{2},$

   we get

   =\int_0^{\pi/3}(1-\cos2\theta)\,d\theta.$$ Hence $$I=\left[\theta-\frac{\sin2\theta}{2}\right]_0^{\pi/3} =\frac{\pi}{3}-\frac{\sin(2\pi/3)}{2}.

   Since

   $\sin\left(\frac{2\pi}{3}\right)=\frac{\sqrt3}{2},$

   we obtain

   $I=\frac{\pi}{3}-\frac{1}{2}\cdot\frac{\sqrt3}{2}=\frac{\pi}{3}-\frac{\sqrt3}{4}.$

10. Match with options

$\boxed{\frac{\pi}{3}-\frac{\sqrt3}{4}}$

This is Option B.