It says in Great Expectations,
The period of exaggerated reaction consequent on all public wrongdoing—and which is always its heaviest and longest punishment—was still far off.
What is this "punishment" referring to?
A fuller quotation sets the context better:
At that time jails were much neglected, and the period of exaggerated reaction consequent on all public wrongdoing—and which is always its heaviest and longest punishment—was still far off. So, felons were not lodged and fed better than soldiers (to say nothing of paupers), and seldom set fire to their prisons with the excusable object of improving the flavour of their soup.
Dickens is telling us that this story is set before there was any improvement to the condition of prisons and inmates. Although Dickens was writing in 1860/61 when there had been much new prison building, between 1842 and 1877 ninety new prisons were built in Britain which still form the core of our system, Great Expectations is set between 1812 and 1840, with Pip's time in London being in the 1820s, much in advance of that building programme.
While there is some ambiguity about the sentence, my reading is that Dickens is saying that the ‘exaggerated reaction’ is the programme of prison reform characterised by the prison building and the Gaols Act of 1823 which stated
that prisons should be made secure; gaolers should be paid; female prisoners should be kept separately from male prisoners; doctors and chaplains should visit prisons and lastly, attempts should be made to reform prisoners. BBC Bitesize
I didn’t initially read Dickens as believing it was an exaggerated response, Dickens was a proponent of Prison reform from his childhood experience of his Father being incarcerated in the Marshalsea for Debt, after all. But further digging suggests that although he was active in campaigning against poverty, and against imprisonment for Debt, he was also very anti what he perceived as poor management of prisons, that wasted money, and against prisoners being better provided for than inmates of Workhouses. I am not able to find any version of his ‘Pet Prisoners’ article from ‘Household Words’ other than this imaged version in Google Books so I hope you will forgive me if I don’t quote extensively from it, copytyping between documents on a small laptop screen isn’t my forte, but in that article he clearly expresses that he is scandalised by the fact that the work conducted by prisoners in the new Pentonville Model Prison not only fails to turn a profit, but comes at a net cost to the public purse. He is outraged that the rations for prisoners vastly outweigh those for workhouse residents and rails against the
astonishing consideration for crime in comparison with want and work.
At his time of writing there was a growing backlash against the reforms, and people were starting to think that prisoners were being mollycoddled as they were now treated better than soldiers and people in workhouses.
by the 1860s public opinion was calling for harsher measures in reaction to an increase in crime which was perceived to come from the 'flood of criminals' released under the penal servitude system. Wikipedia:Penal Reform 19th Century
The use of Public in 'public wrongdoing' is not perhaps only about being publicly seen or known, but the OEDs definition 2a:
Of or relating to the people as a whole; that belongs to, affects, or concerns the community or the nation.
The OED also notes that
The various senses pass into each other by many intermediate shades of meaning. The exact meaning often depends upon the noun qualified; in some expressions the precise sense is unambiguous, but in others more than one sense is vaguely present, and it is difficult to determine in what sense precisely the thing in question was originally called ‘public’. E. g. public worship may be public in senses A. 1a, A. 4a; a public meeting may be regarded as public in senses A. 1a, A. 2a, A. 3a, and A. 4a; and a public examination, originally public in sense A. 2b, is now apprehended as public in sense A. 4a.
The precise definitions for all the definition references are not necessary for understanding that 'public' can be quite nuanced and difficult to pin down to a precise definition, while still being understandable. I think what holds for this use in the Dickens' quote is that the wrongdoing is not just publicly seen, but is of concern to and in some sense committed on behalf of the general public. So my reading is that this passage can be broken down thus:
‘the period of exaggerated reaction consequent on all public wrongdoing’ – time during which there is an over-reaction in compensation of any wrongdoing by the state, of which the public is aware. In this instance, the prisons reforms of the building programme and the Gaols Act of 1823.
‘and which is always its heaviest and longest punishment’ – the compensatory over-reaction is both expensive and not seen as effectual, therefore the ongoing burden on the ratepayer and inequality visited on society is punitive and with no clear end point. This outweighs any political cost there may have been in beginning the reforms.
In stating that this was all ‘still far off’ Dickens is just underlining that the prison reforms had not yet begun.
The comment about prisoners setting fire to their prisons because they don’t like the soup is generally described as being in reference to the Chatham Prison Riots of 1861, which some sources say was triggered by a reduction in rations. Dickens is saying that prisoners who would riot over a reduction in their ration when they already get so much more than soldiers or workhouse inmates are like spoiled children in a tantrum rather than having had a legitimate grievance. It is possible that he was not fully aware of the dreadful conditions which appear to have prevailed in Chatham at that time, but also the subject is vast and I’m not entirely sure I have read enough about it myself to know if Dickens is correct to sneer or if he’s being a bit Daily Mail about it all. (The Daily Mail being representative of tabloid newspapers in the UK which are perpetually in a moral panic about something, and not necessarily rigorously accurate in their reporting.)