Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics
The present work proposes a novel framework for emotion-driven procedural sound generation, termed SONEEG. The framework merges emotional recognition with dynamic sound synthesis to enhance user schooling in interactive digital environments. The framework uses physiological and emotional data to gen...
Saved in:
| Main Authors: | , , , , , , , |
|---|---|
| Other Authors: | |
| Format: | Article |
| Published: |
Politeknik Negeri Padang
2025
|
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1833411272496906240 |
|---|---|
| author | Hariyady H. Ibrahim A.A.A. Teo J. Ajis A.F.M. Ahmad A. Yassin F.M. Salimun C. Weng N.G. |
| author2 | 57824533800 |
| author_facet | 57824533800 Hariyady H. Ibrahim A.A.A. Teo J. Ajis A.F.M. Ahmad A. Yassin F.M. Salimun C. Weng N.G. |
| author_sort | Hariyady H. |
| building | UNITEN Library |
| collection | Institutional Repository |
| content_provider | Universiti Tenaga Nasional |
| content_source | UNITEN Institutional Repository |
| continent | Asia |
| country | Malaysia |
| description | The present work proposes a novel framework for emotion-driven procedural sound generation, termed SONEEG. The framework merges emotional recognition with dynamic sound synthesis to enhance user schooling in interactive digital environments. The framework uses physiological and emotional data to generate emotion-adaptive sound, leveraging datasets like DREAMER and EMOPIA. The primary innovation of this framework is the ability to capture emotions dynamically since we can map them onto a circumplex model of valence and arousal for precise classification. The framework adopts a Transformer-based architecture to synthesize associated sound sequences conditioned on the emotional information. In addition, the framework incorporates a procedural audio generation module employing machine learning approaches: granular and wavetable synthesis and physical modeling to generate adaptive and personalized soundscapes. A user study with 64 subjects evaluated the framework through subjective ratings of sound quality and emotional fidelity. Analysis revealed differences among samples in sound quality, with some samples getting consistently high scores and some getting mixed reviews. While the emotion recognition model reached 70.3% overall accuracy, it performed better at distinguishing between high-arousal emotions but struggled to distinguish between emotions of similar arousal. This framework can be utilized in different fields such as healthcare, education, entertainment, and marketing; real-time emotion recognition can be applied to deliver personalized adaptive experiences. This step includes acquiring multimodal emotion recognition in the future and utilizing physiological data to understand people's emotions better. ? 2024, Politeknik Negeri Padang. All rights reserved. |
| format | Article |
| id | my.uniten.dspace-36863 |
| institution | Universiti Tenaga Nasional |
| publishDate | 2025 |
| publisher | Politeknik Negeri Padang |
| record_format | dspace |
| spelling | my.uniten.dspace-368632025-03-03T15:45:18Z Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics Hariyady H. Ibrahim A.A.A. Teo J. Ajis A.F.M. Ahmad A. Yassin F.M. Salimun C. Weng N.G. 57824533800 23392655000 57201882145 57209978636 55390963300 55056163600 36675707100 59550696100 The present work proposes a novel framework for emotion-driven procedural sound generation, termed SONEEG. The framework merges emotional recognition with dynamic sound synthesis to enhance user schooling in interactive digital environments. The framework uses physiological and emotional data to generate emotion-adaptive sound, leveraging datasets like DREAMER and EMOPIA. The primary innovation of this framework is the ability to capture emotions dynamically since we can map them onto a circumplex model of valence and arousal for precise classification. The framework adopts a Transformer-based architecture to synthesize associated sound sequences conditioned on the emotional information. In addition, the framework incorporates a procedural audio generation module employing machine learning approaches: granular and wavetable synthesis and physical modeling to generate adaptive and personalized soundscapes. A user study with 64 subjects evaluated the framework through subjective ratings of sound quality and emotional fidelity. Analysis revealed differences among samples in sound quality, with some samples getting consistently high scores and some getting mixed reviews. While the emotion recognition model reached 70.3% overall accuracy, it performed better at distinguishing between high-arousal emotions but struggled to distinguish between emotions of similar arousal. This framework can be utilized in different fields such as healthcare, education, entertainment, and marketing; real-time emotion recognition can be applied to deliver personalized adaptive experiences. This step includes acquiring multimodal emotion recognition in the future and utilizing physiological data to understand people's emotions better. ? 2024, Politeknik Negeri Padang. All rights reserved. Final 2025-03-03T07:45:18Z 2025-03-03T07:45:18Z 2024 Article 10.62527/joiv.8.4.3101 2-s2.0-85217489069 https://www.scopus.com/inward/record.uri?eid=2-s2.0-85217489069&doi=10.62527%2fjoiv.8.4.3101&partnerID=40&md5=b3e3164607280418a00baae22670d7a3 https://irepository.uniten.edu.my/handle/123456789/36863 8 4 2479 2488 All Open Access; Gold Open Access Politeknik Negeri Padang Scopus |
| spellingShingle | Hariyady H. Ibrahim A.A.A. Teo J. Ajis A.F.M. Ahmad A. Yassin F.M. Salimun C. Weng N.G. Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics |
| title | Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics |
| title_full | Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics |
| title_fullStr | Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics |
| title_full_unstemmed | Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics |
| title_short | Harmonizing Emotion and Sound: A Novel Framework for Procedural Sound Generation Based on Emotional Dynamics |
| title_sort | harmonizing emotion and sound: a novel framework for procedural sound generation based on emotional dynamics |
| url_provider | http://dspace.uniten.edu.my/ |