Ciencia, Arte y Tecnología
A Máquina Sensível is a project that seeks to investigate how words can shape realities and attempts to understand the relationship between creation and orality, fostering critical connections between cosmogonic narratives and techno-poetic spaces. For this, a multimedia computer system is proposed with which users can interact with audiovisual content through Automatic Speech Recognition (ASR). In addition, it is proposed to implement an emotional analysis of utterances in order to be able to explore, from affective computing, a human-machine relationship mediated by natural language.
Taking into account the artistic and practical nature of the project, the methodology and respective work plan follow a practice-based methodology. We understand that this research has technical, practical, and critical components, instrumentalizing various methods and technologies that allow the researcher/creator to reflect and produce, in an iterative process.
The basic idea is that users can interact with the project using Automatic Speech Recognition. After a command word is recognized and/or a sentence is scored, several action takes place in order to create an audiovisual environment.
To accomplish this, my plan was to use Unity3D and windows automatic speech recognition.
The idea is to be able to recognize some predefined words or phrases in order to be able to interact with some audiovisual content.
The contents are classified into 4 categories (GUI, Primitive, Asset, and Shader).
The system will be always in the HOME environment. The only GUI element would be a reference for HELP.
If the command HELP is recognized, then the instructions will be shown.
In the Instructions, we will have access to see other keywords and understand the logic used to operate the program. Basically, the app can create or destroy elements.
The system has two main methods (CREATE & DESTROY). With these two methods, the program will show or hide the element invokes.
It is possible to instantiate more than one element if they are not from the same category.
The properties of the elements can be controlled using the COLOR keyword and the commands for Spatial Graphic Transformation.
The properties are inherited to all the elements of the same category.
PRESS HERE TO SUBMIT
Here my first test!