Evaluation Scenes

Scenes are defined in the Scenes array of the configuration file. Each entry specifies the visual content type, the Unity scene to load, the audio rendering pipeline, and the associated audio files.

QExE supports four scene configurations, corresponding to the combination of audio pipeline and visual content type:

	CGI	360° Video
Object-based audio	Object-based + CGI	Object-based + 360° Video
Ambisonics audio	Ambisonics + CGI	Ambisonics + 360° Video

Object-based Audio + CGI

Individual mono audio files are associated with spatial audio objects in a CGI Unity scene, rendered through the object-based pipeline.

"Scenes" : [
    {
        "stimuli_ID"        : "Scene_stereo_bongos_interaction",
        "unityScene_ID"     : "_bongos_demo",
        "visualStimuliType" : "CGI",
        "dimensions"        : [ 30.0, 8.0, 30.0 ],
        "multichannelAudio" : {
            "multichannelAudioFile" : null
        },
        "objectAudio" : {
            "objectAudioFolder"  : "audio/_bongos/edited",
            "objectAudioRouting" : [
                {
                    "pathToFile"   : "1_bongos_L.wav",
                    "inputChannel" : 0,
                    "autoplay"     : 1
                },
                {
                    "pathToFile"   : "2_bongos_R.wav",
                    "inputChannel" : 1,
                    "autoplay"     : 1
                }
            ]
        }
    }
]

Key entries:

• stimuli_ID — a label to identify this scene in the results output
• unityScene_ID — the name of the .unity scene file to load in the Unity Agent
• visualStimuliType — set to "CGI" for Unity-authored scenes
• dimensions — the [X, Y, Z] extent of the virtual scene in metres, used for parameter normalisation when mapping object positions to VST inputs
• multichannelAudioFile — set to null when using the object-based pipeline
• objectAudioFolder — path to the folder containing mono object audio files, relative to pathToAudioVideoScenesContent
• objectAudioRouting — array of audio object entries, each specifying:
• pathToFile — filename or relative path of the mono audio file
• inputChannel — VST input channel index for this object
• autoplay — 1 to loop continuously from scene start; 0 to trigger via event

Object-based Audio + 360° Video

A 360° video scene augmented with spatially rendered audio objects. The _videoPlayer Unity scene is used for all 360° video content.

"audioRendering" : {
    ...
    "renderingPipeline" : "Objects",
    ...
},

"Scenes" : [
    {
        "stimuli_ID"        : "Scene_sax_video_audio_object",
        "unityScene_ID"     : "_videoPlayer",
        "visualStimuliType" : "ODV",
        "videoFiles" : [
            {
                "pathToFile" : "saxophone/video/saxophonePlayer.mp4", 
                "video_ID"   : 1
            }
        ],
        "dimensions"        : [ 5, 5, 5 ], 
        "multichannelAudio" : {
            "multichannelAudioFile" : null
        },
        "objectAudio" : {
            "objectAudioFolder"  : "saxophone/objects",
            "objectAudioRouting" : [
                {
                    "pathToFile"   : "saxophone.wav",
                    "inputChannel" : 0,
                    "autoplay"     : 1
                }
            ]
        }
    }
]

• unityScene_ID — always "_videoPlayer" for 360° video scenes
• visualStimuliType — set to "ODV" (omnidirectional video) for 360° content
• videoFiles — a list of video files associated with this scene. Assigning multiple video files to a single scene is an efficient way to generate ACR test items for video quality evaluation, where all other parameters remain constant across items

note

A scene that is recorded via a 360 camera doesn't change in terms of the content that is recorded. However, it CAN change in terms of encoding, resolution, or frame-rate, for example. Therefore, multiple recordings can be defined in the Scene entry, by providing more videoFiles and subsequent pathToFile and index entries.

Ambisonics Audio + 360 Video

A 360° video scene with a multichannel Ambisonics audio file. Object-based audio entries are left empty.

"audioRendering" : {
    ...
    "renderingPipeline" : "Multichannel",
    ...
},

"Scenes" : [
    {
        "stimuli_ID"        : "Scene_sax_video_ambisonics",
        "unityScene_ID"     : "_videoPlayer",
        "visualStimuliType" : "ODV",
        "videoFiles" : [
            {
                "pathToFile" : "saxophone/video/saxophonePlayer.mp4", 
                "video_ID"   : 1
            }
        ],
        "dimensions"        : [ 5, 5, 5 ],
        "multichannelAudio" : {
            "multichannelAudioFile" : "saxophone/ambisonics/4OA_Saxophone.wav"
        },
        "objectAudio" : {
            "objectAudioFolder"  : "null",
            "objectAudioRouting" : []
        }
    }
]

• renderingPipeline — set to "Multichannel" in the audioRendering block
• multichannelAudioFile — path to the Ambisonics audio file, relative to pathToAudioVideoScenesContent
• objectAudio entries are left empty when using the multichannel pipeline; dimensions is also ignored in this mode

Ambisonics Audio + CGI

A CGI Unity scene with a multichannel Ambisonics audio file.

"audioRendering" : {
    ...
    "renderingPipeline" : "Multichannel",
    ...
},

"Scenes" : [
    {
        "stimuli_ID"        : "Scene_city_1OA_audio",
        "unityScene_ID"     : "_city",
        "visualStimuliType" : "CGI",
        "dimensions"        : [ 50.0, 8.0, 50.0 ],
        "multichannelAudio" : {
            "multichannelAudioFile" : "city/ambisonics/4OA_City.wav"
        },
        "objectAudio" : {
            "objectAudioFolder"  : "null",
            "objectAudioRouting" : []
        }
    }
]