[m]\sqrt{1+x^2}=t[/m] ⇒ [m]1+x^2=t^2[/m] ⇒
[m]x^2=t^2-1[/m]
[m]x=\sqrt{t^2-1}[/m]
[m]dx=\frac{2t}{2\sqrt{t^2-1}}dt[/m]
Меняем пределы интегрирования
[m]x_{1}=0 ⇒ t_{1}=1[/m]
[m]x_{2}=\sqrt{3} ⇒ t_{2}=2[/m]
[m] ∫ ^{\sqrt{3}}_{0}x^5\sqrt{1+x^2}dx= ∫ ^{2}_{1}(\sqrt{t^2-1})^{5}\cdot t\cdot \frac{2t}{2\sqrt{t^2-1}}dt=∫ ^{2}_{1}(\sqrt{t^2-1})^{4}\cdot t^2=[/m]
[m]=∫ ^{2}_{1}t^2\cdot (t^2-1)^2=∫ ^{2}_{1}(t^6-2t^4+t^2)dt=\frac{t^7}{7}-\frac{2t^5}{5}+\frac{t^3}{3}=[/m]
[m]=\frac{2^7}{7}-\frac{2\cdot 2^5}{5}+\frac{2^3}{3}-\frac{1}{7}+\frac{2}{5}-\frac{1}{3}[/m]