According to another embodiment of the invention, the multi-modal input may include a string of text, such as keywords, that are received as commands or questions.
![proloquo semantically organized proloquo semantically organized](https://i.pinimg.com/originals/a9/b5/78/a9b578af8c0c3e40707c95690624e4ba.png)
The communication channels allow the system to receive multi-modal input such as text-based commands and questions and/or voice commands and questions. According to one exemplary embodiment of the invention, context information may be maintained in a multi-modal environment by providing communication channels between multi-modal devices, or mobile device, and the system. According to one embodiment of the invention, infrastructure may be provided to maintain context information during multi-modal interactions, such as speech and/or non-speech interactions. This robustness to partial failure promotes the feeling of a natural response to questions and commands. The robustness to partial failure is achieved through the use of probabilistic and fuzzy reasoning at several stages of the process.
Proloquo semantically organized full#
Further, at each step in the process, accommodation may be made for full or partial failure and graceful recovery.
![proloquo semantically organized proloquo semantically organized](https://i1.rgstatic.net/publication/361255474_FAIR_Knowledge_Graphs_with_Semantic_Units-a_Prototype/links/62a70d6c55273755ebe944bc/largepreview.png)
Alternatively, a combination of speech-based and non-speech-based natural language query and response environment may be created. Through this integrated approach, a complete speech-based natural language query and response environment may be created. This environment makes maximum use of context, prior information, domain knowledge, and user specific profile data to achieve a natural environment for one or more users submitting queries or commands to multiple domains.
![proloquo semantically organized proloquo semantically organized](https://maymay.net/maymay/tech/semdrupal/lod-datasets_2008-02-28.png)
In another aspect of the invention, systems are provided for overcoming the deficiencies of prior systems through the application of a complete speech-based information query, retrieval, presentation and command environment or a combination of speech-based and non-speech-based information query, retrieval, presentation and command environment.