$V = (6a - 2x)(6b - 2x)x$
$V = 36abx - 12(a + b)x^2 + 4x^3$
$dV / dx = 36ab - 24(a + b)x + 12x^2 = 0$
$x^2 - 2(a + b)x + 3ab = 0$
$A = 1; \,\, B = -2(a + b); \,\, C = 3ab$
$x = \dfrac{-B \pm \sqrt{B^2 - 4AC}}{2A}$
$x = \dfrac{2(a + b) \pm \sqrt{4(a + b)^2 - 4(1)(3ab)}}{2(1)}$
$x = \dfrac{2(a + b) \pm 2\sqrt{(a^2 + 2ab + b^2) - 3ab}}{2}$
$x = (a + b) + \sqrt{a^2 - ab + b^2} \,\, $ and
$x = (a + b) - \sqrt{a^2 - ab + b^2}$
If a = b:
From
$x = (a + b) + \sqrt{a^2 - ab + b^2}$
$x = (b + b) + \sqrt{b^2 - b^2 + b^2}$
$x = 3b$ (x is equal to ½ of 6b - meaningless)
From
$x = (a + b) - \sqrt{a^2 - ab + b^2}$
$x = (b + b) - \sqrt{b^2 - b^2 + b^2}$
$x = b$ okay
Use $x = a + b - \sqrt{a^2 - ab + b^2}$ answer