Without reading the story itself, just from what you have put in the question (detective talking about a criminal on parole who's committing a crime), I'd think that it isn't just "clemency foolishness". The foolishness here is the short prison time (counting parole) and clemency offered to prisoners when they show repentance — which, naturally, won't sit well with the detectives who put in time and effort into building a case only to see a wheedling criminal back on the streets in no time!
Of course, parole violators usually lose any possibility of further parole or pardon of their sentence, so the detective thinks that if Jimmy is out on parole and has committed a crime, he's going back and not coming out again anytime soon.