This adds the ability to load and animate a mesh for hand tracking on
the Oculus Quest. It is more or less identical to the current
functionality on the vrapi driver.
One key part of this change is that getPose in OpenXR will see if action
spaces are active before locating their spaces. This is due to some
behavior observed on the Oculus Quest with hand tracking where pose
actions for controllers would return invalid data with all of the
location flags erroneously set. The only way to detect and work around
this is to check the pose action state. When this happens, we fall back
to returning the pose of the wrist joint, which is where the Oculus hand
mesh wants to be drawn. In the event that both controllers and hand
tracking are active, the controller pose will be returned by getPose but
the wrist joint can still be accessed using getSkeleton.
Note that this does not yet include support for properly scaling the
hand mesh.
There are numerous opportunities for optimization here that may be
investigated in the future, though performance is well within an
acceptable range right now.
The notdef glyph will get rendered instead, which is slightly better.
Note that the default font does not have a notdef glyph (bug).
Note that notdef will be rasterized multiple times right now.
Currently there is a single allocator function used in arr_t. Its
behavior depends on the values for the pointer and size arguments:
- If pointer is NULL, it should allocate new memory.
- If pointer is non-NULL and size is positive, it should resize memory.
- If size is zero, it should free memory.
All instances of arr_t use realloc for this right now. The problem
is that realloc's behavior is undefined when the size argument is zero.
On Windows and Linux, realloc will free the pointer, but on macOS this
isn't the case. This means that arr_t leaks memory on macOS.
It's best to not rely on undefined behavior like this, so let's instead
use a helper function that behaves the way we want.
The VrApi implementation now checks that X, Y, A, B buttons exist on that
specific controller. X,Y are on left; A,B on the right controller. That
mapping covers Quest Touch and Quest 2 controllers.
Functions to calculate the angle between two vectors. Angle is always
positive. Implementations give the same result as this Lua code:
```lua
local function lua_angle(v1, v2)
return math.acos(v1:dot(v2) / (v1:length() * v2:length()))
end
```
If either vector is zero-length, the pi/2 value is returned.
These functions read an unsigned 32 bit integer from the Lua stack
and error if the value is negative or too big. Currently converting
Lua numbers to integers will silently wrap or invoke undefined behavior
when they are outside of the acceptable range.
For projects that don't want the overhead of type/bounds checks, the
supercharge build option (LOVR_UNCHECKED) can now be used to skip all
type/bounds checks.
Correcting the order of stack operations to fetch RGB components from
the table and to put in conversion the results.
Before the fix these two calls produced different results:
`lovr.math.gammaToLinear( 0.1, 0.2, 0.3 )`
`lovr.math.gammaToLinear( {0.1, 0.2, 0.3} )`
The current flag did not work because float shader flags are not
supported. It was also not very useful because it was per-shader
and did not use the alpha cutoff property of glTF materials.
Instead, let's turn the shader flag into an enable/disable boolean,
and add a scalar material property named "alphacutoff" that gets
read by the glTF importer.
When the alphaCutoff flag is enabled, the material property will be
compared against the pixel's alpha value to decide whether it should
get discarded.
- When calling lovr.graphics.stencil, the color mask is initially
disabled, and gets restored to its initial state afterwards.
- However, when it's restored, it uses lovrGraphicsSetColorMask, which
just sets shadow state that doesn't make it all the way to GL until
another draw is done.
- The consequence of this is that if you call .stencil and then don't do
a draw, any clears that happen will use the old (disabled) color mask,
preventing the color buffer from being cleared.
- The solution here is to lower the color mask change down into opengl.c
where it can directly hit OpenGL.
Vector methods are extended to receive vectors as individual numbers
for each of x,y,z,w component. The vector objects are still supported
as well.
Previously only single value scalar was supported. This change maintains
backward compatibility.
```
v = vec3():add( 1, 2, 3 ) -- both do the same
v = vec3():add( vec3(1, 2, 3) ) _/
v = vec4():mul( 2 ) -- x component is a default for y, z
v = vec4():mul( 2, 2, 2, 2 ) _/
v = vec2():lerp( 2, 2, 0.5 ) -- in lerp, dot, cross, distance
v = vec2():lerp( vec2(2, 2), 0.5 ) _/ all components are mandatory
```
- You were able to write a Blob to a ShaderBlock
- Using ShaderBlock:send(Blob, offset, size)
- It was not flexible enough and it was broken
- The data was read from `offset` bytes into the Blob.
- The data was written to the beginning of the Buffer.
- The Buffer was flushed at `offset` bytes into the Buffer.
- This commit changes the signature of the variant
- to ShaderBlock:send(Blob, srcOffset, dstOffset, size)
- and hopefully fixes the behavior.
- Also why is this entire commit description a bulleted list
If you create and destroy objects quickly (using :release), malloc
might give you the same pointer. When we look up this pointer in
the userdata cache, it'll give you an invalid Proxy/pointer, which
throws an error like "Calling 'fn' on bad self".
When collecting objects, remove them from the userdata cache.
Start is mainly used for setting up graphics-related stuff, since it
was created to perform setup after the window/graphics module is
initialized. Since the display driver is the only one doing rendering,
it makes sense to only call start on the display driver.
...also fixes a bug where start is getting called twice.
- We need some headset initialization to happen upfront
- But we still want some delayed initialization for when graphics is ready
- Go back to headset initialization happening when module is required
- Add lovr.headset.start that can be used for post-graphics init
lovrCheck is a new way of performing runtime assertions.
It's identical to lovrAssert, except it's compiled out if
LOVR_UNCHECKED is defined.
It is meant to be used for non-mission-critical validation, for
example proper usage of types passed to the Lua API. lovrAssert
should still be used to check return values from platform APIs.
- Use incoming depth settings to determine whether depth test should be
enabled or disabled (wtf)
- Always track state.depthTest, even if depth test is disabled
- Previously, animate was converting from oculus basis to lovr basis.
- Not all hand models are animated.
- Instead, apply the compensation in newModel.
- This means that both animated and non-animated models have correct orientation.
- Verified that regular getPose is returning correct rotation as well.
- Previously, animate was converting from oculus basis to lovr basis.
- Not all hand models are animated.
- Instead, apply the compensation in newModel.
- This means that both animated and non-animated models have correct orientation.
- Verified that regular getPose is returning correct rotation as well.
If a Texture is created from a handle, that means someone else created
it, so we expect them to destroy it. We were always destroying handles,
and I guess this was usually okay because glDeleteTextures is idempotent.
However, we're seeing a crash in the Oculus driver when OVR is torn
down. Presumably it is trying to access its swapchain textures after we
destroyed them. Not sure why this wasn't an observable issue before,
maybe it's a new regression. Still, it makes sense to only delete the
GL texture handle if we were the one that created it.
We don't need to check this for the renderbuffer since we always own those.
Audio currently stutters on the wasm build. It is much more severe
in Chrome than in Firefox (very rare/subtle in Firefox). miniaudio
is currently using ScriptProcessorNode, which is deprecated because
it processes audio on the main thread. There's a new API that lets
you programmatically process audio on a thread called AudioWorklet,
but it's hella complicated. miniaudio doesn't want to support this
because it's complicated and requires a separate JavaScript request
but it seems like it would be possible to work around using a Blob.
In the meantime, miniaudio bumps up the buffer size on WebAudio, so
let's just use that in hope that it helps.
- Sources without converters always read into the beginning of the
raw buffer, overwriting previous frames if the source was rewound
due to looping. This resulted in an audible click whenever the
source was rewound.
- After looping, Sources without converters would try to read too
many frames -- they would read a full buffer instead of only the
necessary number of frames.
- A list or map of effects can be provided to newSource.
- false can be used to bypass effects.
- All effects are enabled by default.
- Occlusion-y effects should only take effect when setGeometry is called
- Spatializer is responsible for ensuring this.
ODE errors, debugs and messages are redirected into LOVR's log system
by a callback mechanism for each.
The ODE submodule is updated to revision that does not crash when error
or debug occurs.
30e01f upgraded stb_image to include its 95560b commit from its #960
pull request. This made stb_image fail more aggressively on EOF
conditions when refilling huffman buffers in deflate streams. I think
it might be failing _too_ aggressively, though. We are able to pad our
input compressed buffers since the zip file format is guaranteed to have
extra data at the end (for, e.g., the end of central directory record).
This appears to be sufficient to fix compressed zip archives for the
time being. It's possible that more virtual padding needs to be added,
and it may be good to try to fix this in stb_image itself.
The falloff is the minimum distance at which inverse distance
attenuation takes place.
A non-positive value disables distance attenuation.
In the Lua API, nil can be used to disable attenuation, a boolean can be
used to enable attenuation with a default minimum distance, or a number
can be used for full control over the parameter.
Add support for importing ambisonic WAV files and 24/32 bit PCM WAV files.
The standard ambisonic format used internally in LÖVR is ACN channel ordering with SN3D normalization.
Anything else will be converted to this form.
There are a few restrictions and assumptions:
- Only 1st order ambisonics are supported. They need to have 4 channels.
- They can be in AMB format (Furse-Malham order/normalization), detected via WAVE_EXTENSIBLE GUID.
- Any other 4 channel file is assumed to be in "AmbiX" ACN/SN3D format.
- It seems that most ambisonic files in the wild that claim to be AmbiX are just 4 channel WAVs without any metadata.
- This means that non-ambisonic 4 channel WAVs could ambiguously be mistaken as ambisonic. This is incurred as a limitation of LÖVR.
- Ambisonic files can not currently be played back. SteamAudio currently has numerous bugs with this.
- Perhaps it would be possible to write an ambisonic rotator/panning decoder to use as a default implementation.
- Compute feature requires compute shaders, image load/store, and SSBOs.
- GLSL 330 is always used, instead of changing depending on compute shader extension.
- Explicitly enable compute shaders, image load/store, and SSBO extensions when needed.
This allows implementations that don't support GLSL 430 to run compute shaders,
and keeps the min supported GL version more consistently at GL3.3.
If 64 sources are playing and a new one is started, Source:play will
return false.
Instead of a linked list, a static list of 64 Sources is used.
Bit scanning intrinsics are used to efficiently iterate the list,
using a mask (still deciding on this).
It isn't always correct once you add locomotion to a project, and
it actually becomes harmful because the pose ping-pongs between
the default pose and the user pose, causing distortion.
- If no converter is needed, don't create/use it
- If no spatialization is needed, don't copy
In the best case, samples willi now be read into a buffer and immediately mixed into the output.
This is a large patch which adds a new Oculus Audio spatializer. Oculus Audio is slightly different from the dummy spatializer in a few ways:
- It *must* receive fixed-size input buffers, every time, always.
- It can only handle a fixed number of spatialized sound sources at a time.
- It has a concept of "tails"; the spatialization of a sound can continue after the sound itself ends (eg echo).
Changes to audio.c were needed to support Oculus Audio's quirks:
- audio.c now supports a "fixedBuffer" mode which invokes the generator/spatializer in fixed size chunks
- Each source now has an intptr_t "memo" field that the spatializer may use to store whatever (Oculus spatializer uses this to handle the sound source limit).
- The spatializer interface got a couple new methods: A "tail" method which returns a sound buffer after all sources are processed; and "create" and "destroy" methods that are called when a sound source is created or destroyed (Oculus spatializer uses this to populate/clear the "memo" field).
Along the way some other miscellaneous changes got made:
- lovr.audio.getSpatializerName() returns the current spatializer
- Spatializer init now takes in "config in" and "config out" structs (Spatializer changes fields in config out to request things, currently fixed buffer mode).
- lovr.conf now takes t.audio.spatializer (string name of desired spatializer) and t.audio.spatializerMaxSourcesHint (Spatializers with max sources limits like Oculus will use this as the limit).
- audio.c went back to tracking position/orientation as vectors rather than a matrix
- A file oculus_spatializer_math_shim.h was added containing a minimal copypaste of OVR_CAPI.h from Oculus SDK to support a ovrPoseStatef the spatializer API needs. This may have license consequences but we are probably OK via a combination of fair use and the fact that a user cannot use this header file without accepting Oculus's license through other means.
Some work remains to be done, in particular there is an entire reverb feature I did not touch and LOVR_USE_OCULUS_AUDIO cannot be activated from tup. Oculus Spatializer works better when it has velocity and time information but this patch does not supply it.
* Stop also uninitializes
* Reset doesn't exist. Just stop and start instead.
* lovrAudioInit no longer takes config, and config is now private.
Call lovrAudioStart if you want to start.
* ma_device_{un}init and start/stop are only called from one place each,
reducing the risk of dangling state
* Takes device type, so you only get either playback or capture devices
* Doesn't store devices in state, reducing risk of dangling pointers
* Uses names instead of identifiers, since miniaudio identifiers become
invalid if you call "getDevices" again
* Better diagnostics
* Split up lovrAudioInitDevice to be per-type, cleaner that way
* UseDevice now takes type and name, instead of just identifier
aka a9541579f38a0c1bab4bba294f3602fa0b80f127, plus cherry-pick of
2dc604ecde0f02280690c72f943bfb8bf52dd820.
There is a crasher in 0.10.13 and newer on Oculus Quest
(See https://github.com/mackron/miniaudio/issues/247)
they take the same arguments so we can't overload
the function parameterically.
also I find it pretty confusing that lovr uses
overloads so much in the api,
so I really don't mind having
a separate constructor :S
By looking for failed start and requesting then;
and then emitting a new event type when
permission has been granted or rejected;
and then using that event in the default
boot.lua to re-start capture.
- The lovr.headset.renderTo callback can now be nil, causing an empty frame to be submitted to the compositor.
- lovr.mirror will still be called if lovr.draw is nil. This means the window will be (correctly?) cleared to the background color now if lovr.draw is nil.
This prevents runtimes from thinking the app is missing/unresponsive when lovr.draw is absent. To get the old behavior, just don't call lovr.headset.renderTo.
lookAt() returns view matrix that can be used to transform the camera
perspective. target() returns model matrix that is used to change
model transform. Results are matrix inversions of one another. Now both
functions exist it is possible to use right one and avoid extra matrix
inversion.
- Link against dl (this was probably why ffi.load didn't work)
- Store shared libraries in the apk uncompressed
- Libraries are added during the "package" aapt call instead of "add".
- Because -0 (used to store uncompressed) only works during package.
- classes.dex is added here too now.
- There's only one aapt invocation now, which is kinda nice.
- The lib folder needs to be in a subfolder now ("raw" was chosen).
- Because "package" and "add" subcommands work differently.
- Store shared libraries in the apk page aligned
- Required passing -p to the zipalign invocation.
- This is needed because dlopen("zip!path") needs it.
- android:extractNativeLibs="false" is added to the XML manifest.
- apk sizes are bigger, but disk usage and install time should improve.
- Fix a bug with moving plugin libraries into the lib folder with CMake.
- Use lovrFilesystemGetSource instead of lovrFilesystemGetExecutablePath
to get the proper path to the apk.
- The plugins folder can contain native plugins.
- CMake will build plugins with CMakeLists in them
- They can check the LOVR variable to see if they are being built inside LOVR.
- They can set the LOVR_PLUGIN_TARGETS variable to a list of targets they build.
- If blank, all non-imported targets added in the folder will be used.
- The libraries built by their targets will be moved next to the executable or into the apk.
- The library loader now tries to load libraries next to the executable or in the APK.
- It is "fixed function" now, this may be improved in the future.
- The lovr.filesystem C require path has been removed.
- enet and cjson have been removed. Use plugins.
stb_image's vertical flip flag was not thread safe in the version
of stb_image we were using. We patched stb_image to use a thread
local variable for the flag. stb_image has since been upgraded to
expose a thread local version of the flag, so our patch is no longer
necessary after upgrading.
The CMake flag to enable the thread local patch did not make very much
sense because thread local stuff is unconditionally used elsewhere.
In boot.lua, it assumes that lovr.headset.init will assert if no driver
is available. This was previously only true on the first call to it,
since after it's initialized, it'll just return early and won't assert.
This will later crash since your lua code will now see a lovr.headset
being available, but calling anything in it will crash since
lovrHeadsetDisplayDriver is NULL
After this fix, initialized becomes false before boot sets up the
headset module again, so that the assertion fires correctly.