Step 1 :
Factor out common factor $ \color{blue}{ 4x } $:
$$ 100x^3-240x^2y+144xy^2 = 4x ( 25x^2-60xy+36y^2 ) $$Step 2 :
Note that the polynomial $ 25x^2-60xy+36y^2 $ is a perfect square trinomial, so we will use the following formula.
$$ A^2 - 2AB + B^2 = (A - B)^2 $$In this example we have $ \color{blue}{ A = 5x } $ and $ \color{red}{ B = 6y } $ so,
$$ 25x^2-60xy+36y^2 = ( \color{blue}{ 5x } - \color{red}{ 6y } )^2 $$