Show HN: Audiocube – A 3D DAW for Spatial Audio
audiocube.appI’ve recently released my solo project Audiocube
I wanted to make a 3D DAW, where spatial audio, physics, and virtual acoustics are all directly integrated into the engine.
This makes it easy to create music in 3D, and experiment with new techniques which aren’t possible in traditional DAWs and plugins.
I’d love to get any feedback on this software (Mac/Windows) to make it better.
You can download it for free through the website.
Thanks, Noah
Neat interface the grouped ones with matching neighbor contours
This looks great! How small of an audio buffer have you been able to get down to? Any plans for an API?
I've been developing a VR spatial sound and music app for a few years with the Unity game engine, bypassing the game engine's audio and instead remote controlling Ambisonic VSTs in REAPER. I can achieve low latency with that approach but it's a bit limited because all the tracks and routing need to be setup beforehand. There's probably a way to script it on REAPER but that sounds like an uphill battle. It would be a lot more natural to interface with an audio backend that is organized in terms of audio objects in space.
What I'd like is more flexibility to create and destroy objects on the fly. The VSTs I'm working with don't have any sort of occlusion either. That would be really nice to play with. Meta has released a baked audio raytracing solution for Quest, and that's fun for some situations but the latency is a bit too much for a satisfying virtual instrument.
Here's my project for context: https://musicality.computer/vr
Hey, I’ve got down to under 20ms audio buffer, but it depends on the complexity of the scene.
What you’re working on sounds really cool, I’ll have a look at it!
It sounds like Audiocube offers the kind of features that you need, although it doesn’t have realtime audio input (yet, I’m working on it and have it partially working).
Looking at your project it looks like it would be great to integrate the two somehow - Audiocube would be awesome in vr but I have no vr dev experience
I assume this can do sound rendering, like simulating a conversation on a subway platform while a subway passes by?
Or singing while walking through a tunnel?
Since it has capabilities that would be hard to replicate, rather than show the tool on the landing page, I would show the output. Remove the clutter and force people to listen to what the tool can produce.
The tool as it is now, is being marketed towards yourself, people that wanted to build that tool and know what it is. But everyone will know what it can do after listening to sample output.
Yes it can do all of those things.
You are right about the audio demos, I'll update the frontpage with this soon. Thanks!
You using Blender's viewport? (Great idea if so, nearly perfect recreation if no!)
Aha, yeah blender was a huge UI inspiration, it’s all handmade from scratch though!
Awesome work, you nailed it!
It's funny to see this now because I've been for a couple weeks looking into audio spacialization. After a lot of research and even trying to write my own spatializer plugin, I found that Game Engines have probably the most complete toolset to do this task. (Specifically I'm using Godot with https://valvesoftware.github.io/steam-audio/).
Steam audio is pretty awesome in that regards because it supports HRTF and all the physical based goodies like occlusion/reflection and sound propagation. So you can get really really immersive spatial audio.
The only downside with this solution is that you can't do offline rendering. So my question is:
can Audiocube do offline rendering? seems like it would be one killer feature for my use case.
It sounds like you’re in a similar field to me so it would be great to connect! Totally agree, yeah Audiocube is built in unity as game engines made the most sense for what I wanted to achieve. Steam audio is super cool and was a lot of use to look at for writing my own stuff.
Offline rendering is at the top of my research list currently. It’s not in the app yet. It’s going to take a lot of work to build given that the physics engine calculations are tied to frame rate etc, so there is a lot of work to do - but I’m sure it’s possible.
Thanks!
Really cool! I've been working on a side project that utilizes spatial audio and I've been pleasantly surprised by the quality I'm getting just using the WebAudioAPI HRTF spatialization. I'm sure this is leagues ahead but it was really nice to find that I didn't really need to do anything to get decent spatial audio other than set the panner node to HRTF mode.
Thanks! Yeah this is definitely a lot more complex and deep than the WebAudio API but that’s still a great tool for some situations.
Will there be a Linux version anytime soon!?
I don't have a Linux machine or much experience with it, but I can easily build to it. If you want to help test a Linux build email noah@audiocube.app
Great! Look for a private message.
[flagged]
Bitwig is all right though.
This is great (in theory). During lockdown I got an ambisonic mic (Rode NF-SFW1) and used it to create Dolby Atmos experiences. The workflow - including sending it to Dolby's tool every time - was such a pain. Adding additional 3d elements was especially annoying and limiting.
Unfortunately that's no longer my hobby so can't test this for you but definitely scratches an itch for past me. Nice
That mic looks fun to play with. On their site there’s a link to a bunch of recordings that can be downloaded:
https://library.soundfield.com/
Thanks!
Yeah, frustration with the existing solutions was one of the main reasons I started this project.
I was working in creating immersive audio, and I just found that none of the plugins allowed the level of placement that I wanted. It was all fiddly, and you couldn't easily move the listening position around, while also moving sound sources.
I came from an audio engineering background, not programming, so it took me a while to even learn how to make software! But now I made the tool I always needed.
Very cool project! And well done on doing it without being a software eng!
How long have you been working on it out of interest?
Thanks! Yeah it was a steep learning curve at the start but I’m very confident now.
I started it in 2020 - here’s a blog post about the history https://www.audiocube.app/blog/audiocube-history
I’ve recently got it working on an iPad which was a frustrating grind but I think the concept is a lot of fun on touchscreen!
Lots of cool software was created by domain experts who taught themselves how to program. And quality is more than often is excellent.
YouTuber Benn Jordan would probably get a kick out of this. He's a major audiophile and did a series of ambisonic ambience.
Thanks, I think so too by the looks of his channel, I'll get in touch!
IIUC, I'd be able to take individual closely miked recording on multiple different instruments and mix them into a soundspace, such that when I listen on stereo headphones, I'd be able to "locate" the sounds on a virtual stage?
(asking because I listen to a lot of live jam music in stereo and noticed that they use a stereo mix with a virtual image)
Yes exactly - this was something I wanted to achieve.
For example, you could have an entire orchestra with each instrument close miked - and then arranged in audiocube in a vritual concert hall.
The user could even move around the space for full placement control - or you could simply render it to a binauralized stereo file for more immersive listening on headphones.
I'd like to develop a mobile "listening" app, which isn't for making projects, but listening to them and placing you in the virtual environment with control
That's how normal panning works, yes (or using a stereo pair of ambient mics, same thing but baked into the recording). But that's left-right only, I assume that these Atmos tools can do it in 3D space.
Does that work if you only have a pair of stereo headphones?
I've had quad speakers before (with a "delay" unit for the rear speakers) but I think it would be hard to do 3D in stereo (I know about https://en.wikipedia.org/wiki/3D_audio_effect and HRTFs but I have never noticed any 3d-ness to the sound I'm listening to in games).
Nice work! Can you export for multichannel playback or is it binaural / stereo?
I'm working on multichannel playback, I've had some experimental success, and plan on adding it into the next update. it just needs more testing on a range of audio interfaces etc.
Currently it only exports/plays back in binaural stereo
Ah. I think I misunderstood the purpose of the app. When viewing the main page, I had the impression that the output would be targeting large Atmos multi-speaker installations. Or that it would export to a format that can play spatially on AirPods.
So right now, is it possible to make something in Audiocube and export it to native spatial AirPod sound ?
I am working on multichannel output but it’s a bit of a headache with all the formats and interfaces etc - I want a simple solution, so that needs a bit of research.
Currently I believe apple spatial uses some kind of 5.1 or 7.1 format audio files mixed with a binauralizer to distribute on headphones. I’m building a method to export in surround formats but it needs a bit more research. Currently it exports as a binaural wav that sounds very 3d, but I’ll look more into apple Spatial Audio as I haven’t researched it much
Outputting 5.1 as a DTS-CD (or something I can convert to a DTS-CD) would give me a reason to buy it.
Very interesting! Did you write your own acoustic simulation engine for this?
Yes - while the app itself is built in unity, it doesn't use Unity's native audio simulation. I implemented a custom spatializer and acoustic engine that can simulate reflections, occlusion, and other stuff that wasn't native in unity
> As a standalone app, Audiocube offers tools, workflows, and processes that go beyond the capabilities of VST plugins.
But does it support VST/AU in order to load instruments rather than "samples"?
There is currently no VST/plugin support, although this is something I would like to include later on.
It's just too much development work at this stage - not only the VST integration itself, but building a full MIDI sequencer timeline, editor, etc to support instruments.
Other DAWs do this perfectly well so I'm focusing on the 3D virtual environment moreso than tradtional DAW functions. Audiocube isn't a tool to compete with the likes of Ableton and Logic, but something to run on the side.
Although, if I had a big enough team I would love to give it all of these features. I'm just doing it solo at the moment, so I need to have a narrower focus.
"Why You Should Not Write a DAW" (ADC 2024)
https://www.youtube.com/watch?v=GMlnh6_9aTc
Yeah this is an interesting talk. A full DAW is maybe over ambitious for one developer, but I don’t think vst plugins would be able to achieve what I wanted. For example vsts can easily spatialise individual sounds, but there is no way to then easily move the listening position through the scene, without a disgusting amount of automation - hence building a virtual 3d environment daw with more placement flexibility
... or start with an open source DAW that already has all the rest, and add the placement flexibility you want ...
But I'm speaking out of self-interest here, and anyway for now your output is limited to binaural stereo which is not really a viable target for production workflows at this point.
This seems intriguing but I'm genuinely confused.
It seems like "bakes in" spatial audio to binaural stereo?
But who is the market for that?
I love spatial audio on my AirPods but a big feature is that it moves with my head and can even be customized for my ears.
And I certainly don't want it applied when downsizing to a mono Bluetooth speaker.
It seems like you'd need to export your final product to surround/Atmos for the way people want to, and currently do, consume spatial audio? I assume the target here is Apple Music, short films, etc.?
I mean I think the concept of the 3D DAW is great. I just want to make sure there's a product-market fit here, so you can succeed. Or is there a market I'm overlooking?
Thanks for the intrigue, and yes these are all valid questions and things I'm trying to work out.
I think I need to narrow down the market a bit. Currently I'm aiming more at audio and sound design experimenters rather than the deep technical stuff - although that is a goal later.
I'd like to create a 'listener app' so people can have full immersion, control, and exploration of audio scenes.
The binaural process is more of a high powered, more flexible and controllable version of a binarual plugin, so this would be for making binaural mixes rather than slapping on everything.
I personally think the binauralizarion is one of the best sounding out there, it sounds great on headphones and has excellent sound localisation.
But you are completely right. I'm still working out product market fit, and I think it's why I'm finding a bit of friction reaching the right audience.
Thanks for your thoughts! Any feedback and input is very appreciated
Just guessing, but even setting aside VR and whatever kind of stuff they are doing for theaters like imax, there does seem to be an interest in new kinds of immersive 3d audio/video. The Las Vegas sphere aimed to do this iirc, and the money behind it aimed to develop the IP and open similar venues in different places around the world. Supposedly media that’s under development actually uses a smaller copy of the target venue, so if you can do that with a software suite instead it’s probably a good thing.
I've even heard of them using this kind of tech in theatre performances, so there is definitely a bunch of use cases!
Not spatial audio, but reminded me of audioGL (2012, but a newer video posted in 2024): https://m.youtube.com/@AudioGL/videos
https://news.ycombinator.com/item?id=3579543
There was also a BeOS application with a 3D mixing interface: https://en.wikipedia.org/wiki/3dmiX
I believe the HRTF binauralization of object-based audio does qualify as spatial audio, but there are many different definitions floating around. AudioGL is cool though!
Is there an audio format that stores the 3D origins of each sound, so that you could theoretically play it through Airpods (or some other spatially-aware headphones) and hear the sound change as you tilt your head?
This is probably a very odd and roundabout way of doing something akin to what you are asking for, but my recently released VR game DodgeALL[0] has a form of modding support for (wav/mono) audio files which can be uploaded (via SideQuest) to the headset. These are then assigned to "obstacles" which are launched at the player i.e. you could move your head around and hear spatialized audio files as they fly around the arena. You don't even need special headphones, the ones on the Quest do a good enough job, and Meta's Spatial audio SDK does a wonderful job of spatializing audio. I was originally using SteamAudio[1] but switched mainly due to moving from PC-VR to Standalone.
Sorry if this sounds like some self-promotion - I specifically built this functionality[2] for my previous audio engineer so that he could quickly design and test sounds without the hassle of having to build/install an APK with every new audio file.
[0]: https://www.DodgeALL.com (only for Quest 2/3/3s/Pro headsets)
[1]: https://valvesoftware.github.io/steam-audio/
[2]: https://docs.google.com/document/d/1IFQjkroomHh8FocjfA-a55JP...
Dolby Atmos is literally that, I think there are several physical formats that can support it.
Ambisonics, especially higher-order ambisonics, also is capable of spatial panning, though it encodes the 3D soundfield rather than individual sound sources. Again, there are a few physical formats.
In addition to Dolby Atmos there is MPEG-H 3D Audio.
https://en.wikipedia.org/wiki/MPEG-H_3D_Audio
https://ieeexplore.ieee.org/document/7056445
Another way to do this would be to make an app that uses positional information from the headphones and feeds the sound of your choice into a head related transfer function (HRTF) to give the illusion of spatial audio.
You could also just emulate what the music would sound like with a stereo system and support head tracking with that. dear reality does something like that with their dearVR digital audio workstation plugins.
And then I recall there was this one old Kickstarter project that made headphones with that idea. It had an IR transmitter for head tracking.
I think it might be quite difficult to track head movement with just earbuds and not have the tracking drift, though.
I sort of expected this sort of thing to be a nice use for VR, since it already needs to do the tracking. But VR generally didn’t seem to catch on, so…
I have a home theater which is 5.1 which is great for movies [1] and some music [2]. There are 'virtual surround' systems based on [3] of which Airpods are the most famous. Most of those don't work for me except in the sense that I learn 'sounds like I have a cold' means 'the sound is supposed to be above me'. The exception is my Hololens where if I use the web browser to play music and turn around it really sounds like the music is coming from behind me. I have yet to be impressed by spatial audio on my Meta Quest 3.
[1] Oddly the surround sound soundtracks that impress me the most are old VHS tapes encoded with Dolby Pro Logic
[2] let's see... Close to the Edge by Yes, Supernature by Goldfrapp, Maximum-Minimum by Kraftwerk
[3] https://en.wikipedia.org/wiki/Head-related_transfer_function
I'm working on a mobile app that lets you playback projects created in Audiocube and gives you full control over where your listening position.
It's definitely possible to have head tracking, I think it will just depend on what headphones and hardware offer it. It would be cool to give you that kind of VR listening experience but just on headphones
Something like https://en.wikipedia.org/wiki/Dolby_Atmos and https://en.wikipedia.org/wiki/DTS,_Inc.#DTS:X
For those of us unfamiliar with the term DAW, I assume it’s Digital Audio Workstation.
At first glance I thought of: DEW
https://en.m.wikipedia.org/wiki/Directed-energy_weapon
Digital Audio Workstation: https://en.wikipedia.org/wiki/Digital_audio_workstation
It’s a digital audio workstation.
[dead]
[flagged]
Looks as though they offer “Perpetual” for 99.00
I see "Sale Ending Soon: Lifetime Licences 71% off - only £29!" at the top of the pages. It looks really neat, but my interest was more curiosity. You need to login to download and then login again from the app (for the trial), and I ran into more friction than I had time for. Will bookmark in case I have a need for this later.
Thanks for the feedback, I'll make it easier to get in in the next update! Also you don't need to log in inside the app. Just click log in and then trial mode, you dont actually need to enter details. But yeah it's maybe a bit confusing
[flagged]