How to Use Sound to Make 3D Geometry in Unity

Final product image
What You’ll Be Creating

In this tutorial, you’ll learn how to make a simple audio visualizer in Unity. We’ll go through the code that reads out the audio data, and then through several ways to apply this data inside a Unity game.

You’ll need a basic understanding of Unity to accomplish this. You’ll also need the music file of a song to visualize—go find something on newgrounds.com if you need something fun, like Mission Completed XL.

The project files can be found on GitHub, if you want to check them out, but are not strictly necessary to accomplish this. The .unity3d file we’ll later use is also in there.

Ready? Let’s go!

1. Line Visualizer

Create a new Unity Project, add your music file to the asset folder, then create a new JavaScript file and name it lineVisualizer. Open this file, and add the following code:

public var detail: int = 500;
public var amplitude: float = 0.1;
private var startPosition: float;

function Start()
{
    startPosition = transform.localPosition.y;
}

function Update() 
{
	var info: float[] = new float[detail];
	AudioListener.GetOutputData(info, 0); 
	var packagedData: float = 0.0;
	
	for(var x: int = 0; x < info.Length; x++)
	{
		packagedData += System.Math.Abs(info[x]);	
	}
}

This script will take the available audio data and apply it in certain ways. The detail variable tells it with what “resolution” to read out the audio data. Higher values in this spot can create a more “jittery” result. You can experiment to find out the best setting later on, and leave it at the basic value for now.

Right now, it doesn’t do anything, as we haven’t specified what there is to do, and what to listen to.

Add the following line at the end of the Update() function in the lineVisualizer script:

transform.localPosition.y = startPosition + packagedData * amplitude;

Then, create an sphere, name it LineVisualizer, and add the script to it. Move the LineVisualizer in front of the camera.

We also need to have the sound in the scene, so let’s get to that. Create a new object and name it AudioSource; it will be the “speaker” that creates the music. You can drag the music file from the asset folder on it, and it will begin playing the file immediately upon starting the scene.

Before we try it, though, set the audio component to play immediately and to loop. Also, go to the music file itself and set it to 2D Sound. This will ensure that the music can be heard equally well, wherever the AudioSource is placed. Finally, make a prefab out of it, so that we can use it later.

If everything has been set up correctly, the lineVisualizer should move up and down. Success! The audio data is being read out and applied in a fun and interesting way.

Now let’s make it prettier.

Create an empty object and add a trail renderer to it, and chain it to the lineVisualizer. These trail settings will create a nice effect:

Chain the LineVisualizer to the camera, so that it moves when the camera moves. The setup should look like this:

Next, create a small JavaScript file called rotation.js, and add the following code to it:

#pragma strict

var speed: float = 15.0;

function Update()
{
    transform.Rotate(0, speed * Time.deltaTime, 0);
}

Put the rotation script on the camera, so that it will rotate around itself.

Try it out! The sphere should bounce up and down, and draw a line behind it.

To make it prettier, turn the cube invisible by unchecking its mesh renderer component, and position it a bit lower so that it doesn’t jump out of the screen. Also, set the camera background color to black, add a directional light in the scene, and give the trail a nice color and material.

You should have a simple line visualizer, which will jump and down with the music, and draw a fitting line around. Try it out in this build:

2. Bar Visualizer

Bar visualizers are a bit trickier. We’ll put several “bars” next to each other, which will jump to the music, each in their own way. Getting a “true” bar visualizer done is way more complicated than this rather simple introduction can cover, so we’ll have to create a fake one (which will look good anyway).

In the project files, you’ll find a 3D file with a special cube. It’s not strictly necessary, so you can skip it if you want to.

Create a new cube, and swap out the standard cube mesh for the aforementioned visualizerCube mesh. The new one has its centerpoint at one edge. When we scale up the cube during runtime, it will then only move in one direction.

Create a new script called barVisualizer. Add the following code:

#pragma strict

public var detail: int = 500;
public var minValue: float = 1.0;
public var amplitude: float = 0.1;

private var startScale: Vector3;

function Start()
{
    startScale = transform.localScale;
}

function Update() 
{
	var info: float[] = new float[detail];
	AudioListener.GetOutputData(info, 0); 
	var packagedData: float = 0.0;
	
	for(var x: int = 0; x < info.Length; x++)
	{
		packagedData += System.Math.Abs(info[x]);	
	}

	transform.localScale = new Vector3(minValue, (packagedData * amplitude) + startScale.y, minValue);
}

It’s similar to the lineVisualizer script, but the commands that adjusts the position have been changed. They now adjust the vertical height of the cube.

Create a cube, call it BarVisualizer, give it a nice color, and put the script you just created on it. Then create several copies of it and place them next to each other in front of the camera. It should look like this:

If you try it out, you’ll notice that all cubes move the same way. In order to make each cube grow and shrink like bars, adapt the code to look like this:

#pragma strict

public var detail: int = 500;
public var minValue: float = 1.0;
public var amplitude: float = 0.1;

private var randomAmplitude: float = 1.0;
private var startScale: Vector3;

function Start()
{
    startScale = transform.localScale;
	
	randomAmplitude = Random.Range(0.5, 1.5);
}

function Update() 
{
	var info: float[] = new float[detail];
	AudioListener.GetOutputData(info, 0); 
	var packagedData: float = 0.0;
	
	for(var x: int = 0; x < info.Length; x++)
	{
		packagedData += System.Math.Abs(info[x]);	
	}

	transform.localScale = new Vector3(minValue, (packagedData * amplitude * randomAmplitude) + startScale.y, minValue);
}

This gives each bar its own “strength”, with which it moves. You can see the effect in this build:

3. Visualizing With 3D Scale

This one is a bit different. Instead of adjusting the height of an object, we’ll show the visualization by scaling an entire object up and down. This can be used for background elements in music based games, or to have particles and other effects move to the music.

The code is pretty much the same as the barVisualizer, with the scaling commands enhanced to also scale in three dimensions:

#pragma strict

public var detail: int = 500;
public var minValue: float = 1.0;
public var amplitude: float = 0.1;

private var randomAmplitude: float = 1.0;
private var startScale: Vector3;

function Start()
{
    startScale = transform.localScale;
	
	randomAmplitude = Random.Range(1.0, 3.0);
}

function Update() 
{
	var info: float[] = new float[detail];
	AudioListener.GetOutputData(info, 0); 
	var packagedData: float = 0.0;
	
	for(var x: int = 0; x < info.Length; x++)
	{
		packagedData += System.Math.Abs(info[x]);	
	}
	
	transform.localScale = new Vector3((packagedData * amplitude) + startScale.y, (packagedData * amplitude) + startScale.y, (packagedData * amplitude) + startScale.z);
}

Put it on a sphere (or any other object of your choosing), and create several copies for a nice effect. Try it out in this build:

Conclusion

In this tutorial, you learned how to make a simple, self-contained audio visualizer within Unity, which can be applied in many different ways. You could use these techniques to design psychedelic light shows that move in time to music, or as source data that serves as the basis for an entire game, like the level generation in Audiosurf, where the obstacles you need to avoid appear in time to the beat. Go wild!

{excerpt}
Read More

Leave a Reply

Your email address will not be published. Required fields are marked *