A brief insight in my my singable Text-To-Speech engine

(Hint: Here in this text is the word phone the phonetic phone, and not a phone device. And please do not confuse a phonetic phone with a phonetic phoneme, which is indeed something similar to a phonetic phone)
Today I'm going to tell you about my singable Text-To-Speech engine, which I also integrated into my own DAW. For simplicity's sake, I will primarily explain how MIDI events become the phone information data including f0 fundamental frequency envelopes, which my implementation of the MBROLA algorithm can process directly without re-parsing. And since a company is interested in my technology, where the DAW, which is quite well known, is also implemented in Object Pascal, I will not provide a code snippet this time. Let's start with the explanation of the procedure.

The first step is to extract all lyric text MIDI events from the remaining MIDI events and put them into a separate array, where each lyric text item has three fields, a text UTF8 string, a start time position 64-bit integer and an end time … (read more)

Interval-tree-less interval search on sorted array of linked MIDI events

Today I will tell you a trick to search an linear pre-sorted array of matching linked MIDI events with an interval without using an interval tree.

Each NoteOn MIDI event has a link pointer to its next NoteOff MIDI event, and each NoteOff MIDI event has a link pointer to its previous NoteOn MIDI event. This is important, because we'll stepping back in time while the search on the base on this interlink-information.

First we'll start at the interval start time with a help of a simple binary search, traverse the array until we'll either hit the interval end time or find a interlink-connection-information for stepping back in the time as new interval start time for a new second iteration (only a single next one!) for to get the full requested interval range.

This is now described here in a very simplified way, but as code it would look like this:  https://gist.github.com/BeRo1985/3c50be6480c77dfb320c23f4d88d2f10

The piano roll editor in my DAW project is based on this interval-tree-less interval search of MIDI … (read more)

Arithmetic right shift in Delphi

A little tip if you do need a "Arithmetic shift right" operation in Delphi:

function SARLongint(Value,Shift:longint):longint;
{$if defined(cpu386)}assembler; register;
 mov ecx,edx
 sar eax,cl
{$elseif defined(cpux64)} assembler;
{$if defined(Windows) or defined(Win32) or defined(Win64)}
 mov eax,ecx
 mov ecx,edx
 sar eax,cl
 push rcx
 mov eax,edi
 mov ecx,esi
 sar eax,cl
 pop rcx
 Shift:=Shift and 31;
 result:=(longword(Value) shr Shift) or (longword(longint(longword(0-longword(longword(Value) shr 31)) and longword(0-longword(ord(Shift<>0) and 1)))) shl (32-Shift));

PasVulkan GLTF 2.0 support

I'm making progress with the GLTF 2.0 support in PasVulkan, but I'm still thinking about how to implement everything effectively so that as few draw calls as possible are necessary. The current plan is to pack everything in storage buffers, so that the vertex shaders always have the complete information about everything at any time such as for example morph targets, joints/bones, node transformations, etc. Because then the CPU only needs to update the most necessary and upload to the GPU what is necessary, the rest will be done by the GPU itself then. So stay tuned! 

A happy new year 2019 to you all in advance!

A happy new year 2019 to you all in advance! 😃


SPIR-V reflection and Virtual Reality support at PasVulkan

PasVulkan now has SPIR-V reflection built in directly, without any external third-party dependencies, in other words, it is really pure Object Pascal code that parses the SPIR-V code and extracts information from it.

The new TpVulkanShaderModule.GetReflectionData API parses the SPIR-V code and then returns information about the types, specialization constants and variables used there as a TpvVulkanShaderModuleReflectionData data structure that can be used for example to automatically create Vulkan pipelines and render passes from it.

This is now especially useful for data-driven rendering concepts and dynamic material systems, which will be exactly the next steps at PasVulkan too.

And by the way, PasVulkan now also has Virtual Reality support (which currently uses the VK_KHR_multiview extension and the OpenVR/SteamVR API), see PasVulkan.VirtualReality.pas unit.

PasVulkan have now a working frame-graph API

PasVulkan have now a working frame-graph API with support for multiple GPU-queues, multithreaded cmdbuffer filling, multisampling, multiview rendering and so on.

An initial code example from a real-world project from me: https://gist.github.com/BeRo1985/ab4c97b5e17b84b46b9cecc19db450a9

It was really a lot of work, at least the validation layers didn't have anything to complain about anymore.

It was really a lot of work, so that at least the validation layers didn't have anything to complain about anymore, and that everything was also working. 🙂

And of course with resource lifetime analysis for resource aliasing support to save memory on the memory-usage side.

PasVulkan will get a frame graph feature

I'm currently working on a frame graph feature at PasVulkan, which can simplify a lot when using the Vulkan API, because a frame graph has all important informations about a frame like its compute passes, its render passes, its attachments, its inputs, its outputs, its resources and its lifetimes and so on, so that the frame graph compiler can ideally allocate the GPU resources quite optimally (including reusing/aliasing of GPU resources) and so that the frame graph compiler can generate a quite optimal parallelizable processing-ordering directed acyclic graph, where multiple processing-sequences for each previously created Vulkan GPU queue can be created from it then for more optimal usage of the GPU also.

Here you can find further informations about the frame graph concept:

New PasVulkan subproject named PasGLTF, a GLTF 2.0 loader and writer for the Object Pascal ecosystem

I've a new PasVulkan subproject named PasGLTF, a GLTF 2.0 loader and writer for the Object Pascal ecosystem, including a viewer tool based on PasGLTF.

You can find the project at https://github.com/BeRo1985/pasgltf 

GLSL Astronomy compute shader

Today I present you my GLSL Astronomy compute shader, which I use in my demo productions and games. ☺

// Copyright (C) 2018, Benjamin "BeRo" Rosseaux (benjamin@rosseaux.de) - License: CC0// Hint: It's not super accurate, but it should be good enough for games and demos with sky rendering.#version 430
layout(local_size_x = 1, local_size_y = 1, local_size_z = 1) in;
uniform vec3 tc; // Time constant
layout(std430) buffer ssboAstronomy { vec3 sunPosition; float sunDistance; vec3 moonPosition; float moonDistance; float moonPhase; mat3 celestialTransform; };
const float HalfPI = 1.5707963267948966, // 1.570796326794896619231, PI = 3.141592653589793, // 3.141592653589793238463, PI2 = 6.283185307179586; // 6.283185307179586476925
// The AMD windows GPU drivers do not seem to like constant double precision values here, // so no const prefix in this case.double astronomicalUnitsToKiloMeters = 149597871.0;
const vec2 sinCosOffsets = vec2(0.0, HalfPI);
double(read more)