My Web App
So I have two audio elements that are designed to compare against each other musically, utilizing dbs, gain, fundamental freq., key, harmonics, and pitch comparison. I’ve done this through utilizing the fourier series, using audio analyzers, promixety scores, as well implementing the circle of fifths to make sure to scale across all frequencies.
My Problem
It seems to be going through the whole algorithm but no matter what it keeps giving me a consistent 10 out of 10 and 5 star (my rating system), no matter if I play it with an identical vocal track or and off key key or pitch
The Algorithm (My Code Is Printed Right After)
**Frequency Score:**
fartist: Average frequency of the artist’s chunk
fuserfuser: Average frequency of the user's chunk
Frequency Scorei=max(0.2,1−Frequency Tolerance∣fartist−fuser∣)
The **Harmonic Score** I did the *same thing as above*
kartist: Key of the artist's chunk
kuserkuser: Key of the user's chunk
Distance between kartistkartist and kuserkuser in the Circle of Fifths, denoted as d(kartist,kuser)d(kartist,kuser)
**The Key Proximity Score uses the Circle of Fifths distance:**
Key Proximity Scorei=max(0.2,1−0.1×d(kartist,kuser))
Key Proximity Scorei=max(0.2,1−0.1×d(kartist,kuser))
Chunk Scorei=(0.3×Frequency Scorei)+(0.35×Harmonics Sco
rei)+(0.35×Key Proximity Scorei)
**Final Score:**
Final Score = min((n∑i=1nChunk Scorei)×10,10)
**Weighed values**
Harmonics=35%, Key=35%, Frequency=30%`
My Code
<html>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Audio Visualizer</title>
<style>
body {
margin: 0;
overflow: hidden;
}
@font-face {
font-family: 'Deathcore';
src: url('fonts/Faceless.ttf') format('truetype');
}
@font-face {
font-family: 'Crunk';
src: url('fonts/TheDark.ttf') format('truetype');
}
@font-face {
font-family: 'Sakana';
src: url('fonts/Sakana.ttf') format('truetype');
}
div#content-1 {
display: block;
width: 100%;
height: 100%;
background-color: #000000;
}
div#nav-bar {
position: fixed;
width: 100%;
height: 50px;
background-color: #131313;
border-bottom: solid 1px #ff0000;
}
select {
border-radius: 15px;
border: 2px solid #ffffff;
color: #ffffff;
background-color: #000000;
width: 175px;
height: 36px;
margin-top: 9px;
margin-left: 20px;
}
img#profile-picture {
margin-top: 30px;
width: 350px;
height: 300px;
border: 1px solid #ffffff;
}
audio {
height: 45px;
border-radius: 0px;
border: 2px solid #ffffff;
background-color: #131313;
}
canvas {
width: 48%;
height: 135px;
display: block;
background-color: #000000;
}
#controls {
position: absolute;
bottom: 193px;
right: 30px;
}
button {
margin-left: 5px;
padding: 10px;
background-color: #ff0000;
border: none;
color: white;
cursor: pointer;
border-radius: 5px;
}
button:hover {
background-color: #cc0000;
}
.info {
font-family: Arial;
position: absolute;
width: 320px;
height: 180px; /* Adjusted height to fit the gain */
background-color: #808080;
color: #ffffff;
padding: 10px;
border-radius: 5px;
font-size: 18px;
}
#info-artist {
bottom: 235px;
left: 30px;
}
#info-user {
bottom: 235px;
right: 30px;
}
div#bio {
float: left;
}
div#name-info {
margin-left: 20px;
font-size: 30px;
font-family: Deathcore;
color: #ffffff;
}
div#band-info {
margin-left: 28px;
font-size: 20px;
font-family: Arial;
color: #ffffff;
}
div#lyrics {
position: absolute;
left: 500px;
width: 930px;
height: 350px;
border-radius: 15px;
background-color: #131313;
color: #ffffff;
font-family: Sakana;
font-size: 26px;
}
</style>
</head>
<body>
<div id='content-1'>
<div id='nav-bar'>
<select id='vocal-style'>
<option>Fry</option>
<option>False Chord</option>
<option>Gutterals</option>
</select>
<select id='vocal-artist'>
<option>Danny Worsnop</option>
<option>Alex Terrible</option>
<option>Cj McCreery</option>
</select>
<select id='vocal-song'>
<option>Example 1</option>
<option>Example 2</option>
<option>Example 3</option>
</select>
<button id="generateSampleBtn">Generate New Sample</button>
<button id="downloadSampleBtn">Download Sample</button>
</div>
<br>
<div style='margin-top: 20px;margin-left: 10px;'>
<img id='profile-picture' src='profile_pics/CJ_McCreery.jpg'>
</div>
<div id='bio'>
<div id='name-info'>
Alex Terrible
</div>
<div id='band-info'>
Bands: Slaughter to Prevail
</div>
</div>
<div id="controls">
<button id="recordBtn">Record</button>
<button id="stopBtn" disabled>Stop</button>
<button id="downloadBtn" disabled>Download</button>
<button id="compareBtn">Compare Audio</button>
</div>
<div style='position: absolute; bottom: 5px;width:100%;left: 30px;'>
<canvas style='margin-left: 932px; margin-bottom: -135px;' id='canvas-user'></canvas>
<canvas style='margin-left: 2px;' id='canvas-artist'></canvas>
<audio id='artist-audio' style='width:48%;' src='samples/false_chord/alex_terrible/sample-1.mp3' controls></audio>
<audio id='user-audio' style='width:48%;' src='samples/false_chord/alex_terrible/sample-1.mp3' controls></audio>
</div>
<!-- Updated divs for displaying audio info, including gain -->
<div id="info-artist" class="info">
<strong>Artist Audio:</strong>
<div id="dB-artist">dB: </div>
<div id="fundamental-artist">Fundamental Frequency: </div>
<div id="harmonics-artist">Harmonics: </div>
<div id="pitch-artist">Pitch: </div>
<div id="key-artist">Key: </div>
<div id="gain-artist">Gain: </div>
</div>
<div id="info-user" class="info">
<strong>User Audio:</strong>
<div id="dB-user">dB: </div>
<div id="fundamental-user">Fundamental Frequency: </div>
<div id="harmonics-user">Harmonics: </div>
<div id="pitch-user">Pitch: </div>
<div id="key-user">Key: </div>
<div id="gain-user">Gain: </div>
</div>
<div id='lyrics'>
<div style='margin-top: 20px;'>
<center>Hell is right here! You've been abandoned, you've been abandoned! Hell is right here! Kingdom of storm and thunder</center>
<br>
<div id='score-result'></div>
</div>
</div>
</div>
<script>
window.onload = function () {
let isRecording = false;
let mediaRecorder;
let recordedChunks = [];
const recordBtn = document.getElementById('recordBtn');
const stopBtn = document.getElementById('stopBtn');
const downloadBtn = document.getElementById('downloadBtn');
const userAudio = document.getElementById('user-audio');
const artistAudio = document.getElementById('artist-audio');
const canvasArtist = document.getElementById('canvas-artist');
const canvasUser = document.getElementById('canvas-user');
const ctxArtist = canvasArtist.getContext('2d');
const ctxUser = canvasUser.getContext('2d');
const audioContextArtist = new (window.AudioContext || window.webkitAudioContext)();
const analyserArtist = audioContextArtist.createAnalyser();
const sourceArtist = audioContextArtist.createMediaElementSource(artistAudio);
sourceArtist.connect(analyserArtist);
analyserArtist.connect(audioContextArtist.destination);
const bufferLengthArtist = analyserArtist.frequencyBinCount;
const dataArrayArtist = new Uint8Array(bufferLengthArtist);
analyserArtist.fftSize = 2048;
const audioContextUser = new (window.AudioContext || window.webkitAudioContext)();
const analyserUser = audioContextUser.createAnalyser();
const sourceUser = audioContextUser.createMediaElementSource(userAudio);
sourceUser.connect(analyserUser);
analyserUser.connect(audioContextUser.destination);
const bufferLengthUser = analyserUser.frequencyBinCount;
const dataArrayUser = new Uint8Array(bufferLengthUser);
analyserUser.fftSize = 2048;
let isArtistPlaying = false;
let isUserPlaying = false;
function calculateGain(dataArray) {
return Math.max(...dataArray) / 255;
}
function calculateAudioSpecs(analyser, dataArray, bufferLength, infoPrefix) {
analyser.getByteFrequencyData(dataArray);
let sum = 0;
for (let i = 0; i < bufferLength; i++) {
sum += dataArray[i] ** 2;
}
const rms = Math.sqrt(sum / bufferLength);
const dB = 20 * Math.log10(rms / 255);
document.getElementById(`dB-${infoPrefix}`).innerText = `dB: ${dB.toFixed(2)}`;
let maxIndex = 0;
let maxValue = -Infinity;
for (let i = 0; i < bufferLength; i++) {
if (dataArray[i] > maxValue) {
maxValue = dataArray[i];
maxIndex = i;
}
}
const fundamentalFreq = maxIndex * analyser.context.sampleRate / analyser.fftSize;
document.getElementById(`fundamental-${infoPrefix}`).innerText = `Fundamental Frequency: ${fundamentalFreq.toFixed(2)} Hz`;
const harmonics = maxValue / 255;
document.getElementById(`harmonics-${infoPrefix}`).innerText = `Harmonics: ${harmonics.toFixed(2)}`;
const pitch = fundamentalFreq;
document.getElementById(`pitch-${infoPrefix}`).innerText = `Pitch: ${pitch.toFixed(2)} Hz`;
const key = determineKey(fundamentalFreq);
document.getElementById(`key-${infoPrefix}`).innerText = `Key: ${key}`;
const gain = calculateGain(dataArray);
document.getElementById(`gain-${infoPrefix}`).innerText = `Gain: ${gain.toFixed(2)}`;
}
const circleOfFifths = ["C", "G", "D", "A", "E", "B", "F#", "C#", "G#", "D#", "A#", "F"];
function determineKey(frequency) {
const keys = [
{ note: "C", frequency: 261.63 }, { note: "C#", frequency: 277.18 }, { note: "D", frequency: 293.66 },
{ note: "D#", frequency: 311.13 }, { note: "E", frequency: 329.63 }, { note: "F", frequency: 349.23 },
{ note: "F#", frequency: 369.99 }, { note: "G", frequency: 392.00 }, { note: "G#", frequency: 415.30 },
{ note: "A", frequency: 440.00 }, { note: "A#", frequency: 466.16 }, { note: "B", frequency: 493.88 }
];
let closestKey = keys[0].note;
let closestFreqDiff = Math.abs(frequency - keys[0].frequency);
keys.forEach(key => {
const freqDiff = Math.abs(frequency - key.frequency);
if (freqDiff < closestFreqDiff) {
closestFreqDiff = freqDiff;
closestKey = key.note;
}
});
return closestKey;
}
function getKeyProximityScore(artistKey, userKey) {
const artistIndex = circleOfFifths.indexOf(artistKey);
const userIndex = circleOfFifths.indexOf(userKey);
if (artistIndex === -1 || userIndex === -1) return 0.2;
const distance = Math.min(
Math.abs(artistIndex - userIndex),
12 - Math.abs(artistIndex - userIndex)
);
return Math.max(0.2, 1 - distance * 0.1);
}
function draw() {
if (isArtistPlaying) {
ctxArtist.clearRect(0, 0, canvasArtist.width, canvasArtist.height);
analyserArtist.getByteFrequencyData(dataArrayArtist);
ctxArtist.fillStyle = '#ff0000';
for (let i = 0; i < bufferLengthArtist; i++) {
const barHeight = dataArrayArtist[i] / 2;
ctxArtist.fillRect(i * 2, canvasArtist.height - barHeight, 1, barHeight);
}
calculateAudioSpecs(analyserArtist, dataArrayArtist, bufferLengthArtist, 'artist');
}
if (isUserPlaying) {
ctxUser.clearRect(0, 0, canvasUser.width, canvasUser.height);
analyserUser.getByteFrequencyData(dataArrayUser);
ctxUser.fillStyle = '#ff0000';
for (let i = 0; i < bufferLengthUser; i++) {
const barHeight = dataArrayUser[i] / 2;
ctxUser.fillRect(i * 2, canvasUser.height - barHeight, 1, barHeight);
}
calculateAudioSpecs(analyserUser, dataArrayUser, bufferLengthUser, 'user');
}
if (isArtistPlaying || isUserPlaying) {
requestAnimationFrame(draw);
}
}
artistAudio.addEventListener('play', function () {
isArtistPlaying = true;
draw();
});
artistAudio.addEventListener('pause', function () {
isArtistPlaying = false;
});
artistAudio.addEventListener('ended', function () {
ctxArtist.clearRect(0, 0, canvasArtist.width, canvasArtist.height);
isArtistPlaying = false;
});
userAudio.addEventListener('play', function () {
isUserPlaying = true;
draw();
});
userAudio.addEventListener('pause', function () {
isUserPlaying = false;
});
userAudio.addEventListener('ended', function () {
ctxUser.clearRect(0, 0, canvasUser.width, canvasUser.height);
isUserPlaying = false;
});
recordBtn.addEventListener('click', function () {
recordedChunks = [];
navigator.mediaDevices.getUserMedia({ audio: true }).then(stream => {
mediaRecorder = new MediaRecorder(stream);
mediaRecorder.start();
isRecording = true;
recordBtn.disabled = true;
stopBtn.disabled = false;
mediaRecorder.ondataavailable = function (event) {
recordedChunks.push(event.data);
};
mediaRecorder.onstop = function () {
const blob = new Blob(recordedChunks, { type: 'audio/webm' });
userAudio.src = URL.createObjectURL(blob);
downloadBtn.disabled = false;
};
});
});
stopBtn.addEventListener('click', function () {
if (isRecording) {
mediaRecorder.stop();
isRecording = false;
recordBtn.disabled = false;
stopBtn.disabled = true;
}
});
downloadBtn.addEventListener('click', function () {
const blob = new Blob(recordedChunks, { type: 'audio/webm' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.style.display = 'none';
a.href = url;
a.download = 'recording.webm';
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
});
};
let artistAudioChunks = [];
let userAudioChunks = [];
let isComparisonReady = false;
function resetAudioChunksForComparison() {
artistAudioChunks = [];
userAudioChunks = [];
isComparisonReady = false;
console.log("Chunks reset for new comparison.");
}
function processAudioChunks(analyser, bufferLength, dataArray) {
const chunks = [];
const interval = setInterval(() => {
analyser.getByteFrequencyData(dataArray);
chunks.push([...dataArray]);
}, 100);
return { stop: () => clearInterval(interval), getChunks: () => chunks };
}
function analyzeArtistAudio() {
const artistAudio = document.getElementById('artist-audio');
const context = new (window.AudioContext || window.webkitAudioContext)();
const analyser = context.createAnalyser();
const source = context.createMediaElementSource(artistAudio);
source.connect(analyser);
analyser.connect(context.destination);
const processor = processAudioChunks(analyser, analyser.frequencyBinCount, new Uint8Array(analyser.frequencyBinCount));
artistAudio.addEventListener('ended', () => {
processor.stop();
artistAudioChunks = processor.getChunks();
console.log("Artist audio processing complete. Chunks collected:", artistAudioChunks.length);
if (userAudioChunks.length > 0) isComparisonReady = true;
});
}
function analyzeUserAudio() {
const userAudio = document.getElementById('user-audio');
const context = new (window.AudioContext || window.webkitAudioContext)();
const analyser = context.createAnalyser();
const source = context.createMediaElementSource(userAudio);
source.connect(analyser);
analyser.connect(context.destination);
const processor = processAudioChunks(analyser, analyser.frequencyBinCount, new Uint8Array(analyser.frequencyBinCount));
userAudio.addEventListener('ended', () => {
processor.stop();
userAudioChunks = processor.getChunks();
console.log("User audio processing complete. Chunks collected:", userAudioChunks.length);
if (artistAudioChunks.length > 0) isComparisonReady = true;
});
}
// Circle of Fifths array
const circleOfFifths = ["C", "G", "D", "A", "E", "B", "F#", "C#", "G#", "D#", "A#", "F"];
// Function to calculate key proximity based on the Circle of Fifths
function getKeyProximityScore(artistKey, userKey) {
const artistIndex = circleOfFifths.indexOf(artistKey);
const userIndex = circleOfFifths.indexOf(userKey);
// If either key is not found in the Circle of Fifths, return a minimal score
if (artistIndex === -1 || userIndex === -1) return 0.2;
// Calculate the minimum distance on the Circle of Fifths (wrap-around)
const distance = Math.min(
Math.abs(artistIndex - userIndex),
12 - Math.abs(artistIndex - userIndex)
);
// Scale proximity to range [1.0, 0.2] based on distance (exact match = 1.0, max distance = 0.2)
return Math.max(0.2, 1 - distance * 0.1);
}
// Function to determine musical key from frequency
function determineKey(frequency) {
const keys = [
{ note: "C", frequency: 261.63 }, { note: "C#", frequency: 277.18 }, { note: "D", frequency: 293.66 },
{ note: "D#", frequency: 311.13 }, { note: "E", frequency: 329.63 }, { note: "F", frequency: 349.23 },
{ note: "F#", frequency: 369.99 }, { note: "G", frequency: 392.00 }, { note: "G#", frequency: 415.30 },
{ note: "A", frequency: 440.00 }, { note: "A#", frequency: 466.16 }, { note: "B", frequency: 493.88 }
];
let closestKey = keys[0].note;
let closestFreqDiff = Math.abs(frequency - keys[0].frequency);
keys.forEach(key => {
const freqDiff = Math.abs(frequency - key.frequency);
if (freqDiff < closestFreqDiff) {
closestFreqDiff = freqDiff;
closestKey = key.note;
}
});
return closestKey;
}
function calculateFinalScore() {
if (!isComparisonReady) {
alert('Play both audios completely before comparing.');
return;
}
let totalFrequencyScore = 0, totalHarmonicsScore = 0, totalKeyScore = 0;
const minChunks = Math.min(artistAudioChunks.length, userAudioChunks.length);
const minChunkScore = 0.2;
for (let i = 0; i < minChunks; i++) {
const artistData = artistAudioChunks[i];
const userData = userAudioChunks[i];
const avgArtistFreq = artistData.reduce((a, b) => a + b, 0) / artistData.length;
const avgUserFreq = userData.reduce((a, b) => a + b, 0) / userData.length;
const freqDiff = Math.abs(avgArtistFreq - avgUserFreq);
let freqScore = Math.max(0, 1 - (freqDiff / 1500));
freqScore = Math.max(freqScore, minChunkScore);
totalFrequencyScore += freqScore * 0.3;
const artistHarmonics = Math.max(...artistData) / 255;
const userHarmonics = Math.max(...userData) / 255;
const harmonicsDiff = Math.abs(artistHarmonics - userHarmonics);
let harmonicsScore = Math.max(0, 1 - (harmonicsDiff / 1.5));
harmonicsScore = Math.max(harmonicsScore, minChunkScore);
totalHarmonicsScore += harmonicsScore * 0.35;
const artistKey = determineKey(avgArtistFreq);
const userKey = determineKey(avgUserFreq);
const keyProximityScore = getKeyProximityScore(artistKey, userKey);
totalKeyScore += Math.max(keyProximityScore, minChunkScore) * 0.35;
}
const finalScore = Math.min(Math.ceil((totalFrequencyScore + totalHarmonicsScore + totalKeyScore) / minChunks * 10), 10);
displayFinalScore(finalScore);
}
function displayFinalScore(score) {
let starRating = '★☆☆☆☆';
if (score >= 9) starRating = '★★★★★';
else if (score >= 7) starRating = '★★★★☆';
else if (score >= 5) starRating = '★★★☆☆';
else if (score >= 3) starRating = '★★☆☆☆';
document.getElementById('score-result').innerHTML = `Score: ${score}/10 <br> Rating: ${starRating}`;
}
document.getElementById('compareBtn').addEventListener('click', calculateFinalScore);
document.getElementById('artist-audio').addEventListener('play', analyzeArtistAudio);
document.getElementById('user-audio').addEventListener('play', analyzeUserAudio);
</script>
</body>
</html>
I’ve tried alerting and logging throughout all of my code, I’ve adjusted the weights and normalization of the calculations but in my opinion I think it’s the way the scoring system is handling everything because no matter what I adjust the Score stays at a 10/10 5-star rating.
I’ve tried looking for errors and logs, none found and usually appears everything is running
I’ve tried messing around using and comparing identical audio files as well as different samples that don’t match, and my own recordings. So when I’m hitting notes that don’t match the vocal track I still recieve a high score.
What I’m expecting is, I’m expecting that notes that are matching or close to matching as well as harmonics, and frequency will result in a higher score while the opposite resulting in a lower score based off the algorithm.