You ask:
in essentially inventing his own language, without clear rules of grammar or expression, how did Artaud believe this act of creativity was somehow understandable to others? Or to put it another way, if a literary expression is in a language that cannot be understood, how did he feel it had any value as a communicative or artistic device?
But as Laura Wetherington explains in the article you cite, Artaud believed that his work had "value as a communicative or artistic device" precisely because it eschewed "clear rules or grammar or expression". Or to put it another way, Artaud broke all the rules of communication in an attempt to communicate the incommunicable.
Wetherington writes:
Artaud has spent most of his life trying to escape the strictures of reality as we understand it—reaching for an expression that is beyond language. Carroll’s text was the springboard for creating new words that pointed nowhere and everywhere at once, but Carroll’s work didn’t go far enough because most of the nonsense still had a correspondence to the real. ... Artaud’s translation empties the realism out of Carroll’s magical realism. What we’re left with is magic.
Wetherington, Laura. "Artaud Through the Looking Glass." Jacket2. https://jacket2.org/commentary/artaud-through-looking-glass. Retrieved 26 November 2023.
When Carroll invents a portmanteau word like slithy, it may seem as though the word lacks any referent; but when Humpty Dumpty explains that the word is a compound of lithe and slimy, what seemed like a departure from everyday language becomes reassimilated into ordinary structures of meaning.
Artaud sought to go beyond Carroll. He did not seek to use language in a representational manner. Instead, he sought to expose its limitations as a communicative medium. In a 1932 letter, he wrote:
For I make it my principle that words do not mean everything and that by their nature and defining character, fixed once and for all, they arrest and paralyze thought instead of permitting it and fostering its development.
Artaud, Antonin. "Letters on Language, Second Letter: To J. P., Paris, September 28, 1932." The Theatre and its Double. Translated from the French by Mary Caroline Richards. New York: Grove, 1958. p. 110. Retrieved from archive.org 26 November 2023.
Naomi Greene explains Artaud's stance as follows, translating the same passage differently:
Ideas and sentiments are normally circumscribed by the words expressing them, especially since words, meant for general use and communication, can never exactly correspond to the inner state of each individual. Lacking the words to describe our fundamental states of being, we never become fully aware of our deepest reality. "Therefore I maintain that theoretically words cannot express everything and that because of their predetermined nature, fixed once and for all, they hinder and paralyze thought instead of permitting and aiding its development." Artaud's ideas concerning the relation between a cultural or spiritual transformation of man and a linguistic revolution are clear: in order to achieve a spiritual revolution we must first understand the true nature of our being—an understanding that must be preceded by a radical change in our language.
Greene, Naomi. “Antonin Artaud: Metaphysical Revolutionary.” Yale French Studies, no. 39, 1967, pp. 188–97. JSTOR, https://doi.org/10.2307/2929492. Accessed 27 November 2023. p. 191.
The passage translated in two different versions above reads, in the original French:
Car je pose en principe que les mots ne veulent pas tout dire et que par nature et à cause de leur caractère déterminé, fixé une fois pur toutes, ils arrêttent et paralysent la pensée au lieu d'en permettre, et d'en favoriser le développement.
Artaud, Antonin. "Le Théâtre et son double," Ouvres Complètes vol. 4. Paris: Gallimard, 1964. p. 132. Quoted in Greene, ibid.
The argument of Wetherington and the other scholars she cites can be summarized as follows: In his version of "Jabberwocky", Artaud was tying to get beyond a limited and paralyzing language that relies for its meaning on a correspondence with phenomenal reality. Instead, Artaud puts forward a form of glossolalia. Just as in Pentecostal Christianity, the act of speaking in tongues is taken as a sign that the Holy Spirit is speaking through the speaker, Artaud's nonsense syllables are a stand-in for a reality that goes beyond what ordinary language can express. Any attempt to parse the referential meaning of glossolalic utterances is beside the point; the utterances themselves are evidence of something beyond our frame of reference.
Seen in this way, Artaud's translation of Through the Looking Glass is not a failure, nor is it merely the effluent of a diseased mind. Rather, it is something like Joyce's Finnegan's Wake: the capstone to an artistic career that consistently sought to push art and language to the limits in an attempt to represent the unrepresentable.