Interpret the integral

The expression in the question appears to have a typographical issue. The standard form consistent with the answer choices is: $\int \sqrt{\sec 2x-1}\,dx$ and we must match it to $\alpha \log_e\left|\cos 2x+\beta+\sqrt{\cos 2x\,(1+\cos 2x)}\right|+C$ where the printed part $\cos \frac{1}{\beta}x$ is evidently corrupted. We compute the integral directly and identify $\alpha,\beta$.

Simplify the integrand

$\sqrt{\sec 2x-1}=\sqrt{\frac{1-\cos 2x}{\cos 2x}}$

Using $1-\cos 2x=2\sin^2 x,\qquad \cos 2x$ we get $\sqrt{\sec 2x-1}=\sqrt{\frac{2\sin^2 x}{\cos 2x}}$

A better substitution is to use $t=\sqrt{\cos 2x}$ so that $t^2=\cos 2x$

Substitute $t=\sqrt{\cos 2x}$

Differentiate: $2t\,dt=-2\sin 2x\,dx$ $t\,dt=-\sin 2x\,dx$

Also, $\sin^2 2x=1-\cos^2 2x=1-t^4$

Now $\sqrt{\sec 2x-1}=\sqrt{\frac{1-t^2}{t^2}}=\frac{\sqrt{1-t^2}}{t}$

Hence $I=\int \frac{\sqrt{1-t^2}}{t}\,dx$

From $dx=-\frac{t\,dt}{\sin 2x}=-\frac{t\,dt}{\sqrt{1-t^4}}$ and since $1-t^4=(1-t^2)(1+t^2),$ we obtain $I=\int \frac{\sqrt{1-t^2}}{t}\left(-\frac{t\,dt}{\sqrt{(1-t^2)(1+t^2)}}\right)$ $I=-\int \frac{dt}{\sqrt{1+t^2}}$

Integrate

$I=-\sinh^{-1}(t)+C$

Using $\sinh^{-1}(t)=\log\left|t+\sqrt{1+t^2}\right|,$ we get $I=-\log\left|t+\sqrt{1+t^2}\right|+C$

Substituting back $t=\sqrt{\cos 2x}$, $I=-\log\left|\sqrt{\cos 2x}+\sqrt{1+\cos 2x}\right|+C$

Convert to the required logarithmic form

Multiply inside the logarithm by $\sqrt{\cos 2x}$:

Compute $\beta-\alpha$

$\beta-\alpha=0-(-1)=1$

Comparison with stored answer

Derived answer = $1$.

Stored correct answer = $1$.

They agree.