Our conscious thought, at least at times, seems suffused with language. We may experience thinking as if we were ‘talking in our head’, thus using inner speech to verbalize, e.g., our premises, lemmas, and conclusions. I take inner speech to be part of a larger phenomenon I call inner semiotics, where inner semiotics involves the subjective experience of expressions in a semiotic (or symbol) system absent the overt articulation of the expressions. In this paper, I argue that inner semiotics allows us to bootstrap our way into entertaining thoughts about exact numbers and quantities that we couldn't prior to our competence with a numeric code. I establish that our arithmetic thoughts literally occur as (internal ‘articulations’ of) expressions in a numeric code. However, a problem arises for my view: just as we can slip in overt speech, producing an utterance that deviates from what we mean to say, there is very good evidence that we can slip in inner speech as well. If our arithmetic thought occurs in a numeric code, it's far from clear how we determine when a covert utterance constitutes a slip. In closing, I provide an account of what makes an inner speech utterance a slip.