Dickens is describing Pip's first encounter with a convict, Magwitch.
in a coarse gray
This is shorthand for "coarse gray cloth". It is uncommon, but not unfamiliar, in English to describe clothes by the cut of their cloth. So you might say of a wealthy lady that she was "in a fur" or "in furs" to mean a fur coat.
In fact, I think you're either misquoting the line or quoting it from a mistranslation. I believe the original is:
A fearful man, all in coarse grey, with a great iron on his leg.
In which case the meaning of "coarse grey" is the same, but it sounds a little more forgiving to the modern ear.
The "coarse gray" here indicates a prison uniform. "Coarse" means the cloth is of poor quality, rough against the skin. Gray was the colour worn by most convicts at the time.
a great iron on his leg
This indicates that the man is shackled: he is wearing a heavy iron manacle and/or chain on his leg to slow him down. The shorthand is similar to the one used above: the reader can draw on the context to work out what the author means.
One does not normally wear iron on the body, let alone the singular "a great iron", so we interpret it as meaning a single large piece of metal. Together with the prison uniform, the reader can now work out that this is an escaped prisoner.
Dickens is trying to paint Magwitch as he might be seen by a young boy such as Pip, emphasising the strange and scary aspects of his person. He uses many negative words: Magwitch is "fearful" in appearance, wears "coarse" cloth and comes shackled with "great iron". The odd turn of speech used in the sentence heightens the sense of fear being communicated to the reader.