Interaction is a continuous process of perceiving external information and giving appropriate feedback. External information includes voice, text, image, image, video, etc. In the medical scenario, the information needs to be perceived currently includes audio of doctor-patient dialogue, medical images, medical documents, doctor-patient emotions, EEG signals, ECG signals, etc. The traditional way of using and modeling separately cannot effectively build the internal relations of multimodal information. iFLYHealth proposes a new generation of general-purpose framework for perceptual modeling based on Encoder-Decoder, which solves the problems such as information loss caused by separate modeling in the traditional framework, complicated manual annotation for the training process, and the inability to mix types of information for modeling.