Hamlet says that if the ghost turns its gaze on him, then he will be moved to pity, he will lose his fixed resolve (to avenge his father), and he will weep tears instead of taking bloody revenge.
In “want true colour”, the senses that we need are:
want, v. 1.a. Not to have, to be without; to be deficient in; to lack.
true, adj. 1.d. Of an object, material, or its condition: not liable to break or give way; firm; reliable; sound. Also of a colour: not liable to alter or fade; fixed. Obsolete.
colour, n. 11. General character or disposition; nature, kind.
Oxford English Dictionary.
In “convert my stern effects”, “convert” means “turn from a course of conduct, purpose, disposition, etc.” but “effects” is not so clear. In context it needs to mean the course of conduct etc. that he would be converted (turned) from, but none of the senses of “effect” seem quite right. Singer conjectured that we should read “affects” instead:
We should certainly read affects, i.e. dispositions, affections of the mind: as in that disputed passage of Othello:—‘the young affects in me defunct.’
It is remarkable that we have the same error in Measure for Measure, Act iii. Sc. i.—
‘Thou are not certain,
For thy complexion shifts to strange effects,
After the moon.’
Dr. Johnson saw the error in that play, and proposed to read affects. But the present passage has escaped observation. The ‘piteous action’ of the ghost could not alter things already effected, but might move Hamlet to a less stern mood of mind.
Samuel Weller Singer, ed. (1836). The Dramatic Works and Poems of William Shakespeare, volume 1, p. 496. New York: George Dearborn.
The OED notes that “effect” was a spelling of “affect” in the 1600s, so we don’t have to suppose an error or misprint here, just an uncommon spelling.
Under this interpretation, “stern effects” and “true colour” have similar meanings: “stern” and “true” both mean “fixed, resolute” and “effect” and “colour” both mean “disposition”. This repetition of a idea in different words is a technique that Shakespeare often uses, especially when (as here) he has given the idea a difficult expression.