Recent studies have reported that sensory cortices process more than one sensory modality, challenging the long-lasting concept that they process only one. However, both the identity of these multi-modal responses and whether they contribute to perceptual judgments is unclear. We recorded from single neurons in somatosensory cortices and primary auditory cortex while trained monkeys discriminated, on interleaved trials, either between two tactile flutter stimuli or between two acoustic flutter stimuli, and during discrimination sets that combined these two sensory modalities. We found neurons in these sensory cortices that responded to stimuli that are not of their principal sensory modality during these tasks. However, the identity of the stimulus could only be decoded from responses to their principal sensory modality during the stimulation periods and not during the processing steps that link sensation and decision making. These results suggest that multimodal encoding and perceptual judgments in these tasks occur outside the sensory cortices studied here.