Project Overview

As a qualified piano teacher, I have identified that there are many parts of music teaching that are "not optimal". It is almost impossible to identify every single mistake a player makes in a long piece. This leaves students practising their mistakes, and thus reinforcing bad habits.

This lack of consistency was the problem I wanted to tackle for my TECS project. By the end of the 4 weeks, I aimed to create a full working prototype that catches errors as a pianist plays them in real time, and thus identify mistakes sooner and help students improve faster.


Project Development Journey

TECS is a competition that takes place over 4 weeks. At the end of every week, you give an update of what progress you have made. The person who makes the most progress is deemed the winner.

Week-by-Week Development

  • Week 1: Research phase where I explored audio processing techniques and feasibility studies
  • Week 2: MIDI implementation and basic note detection algorithms
  • Week 3: Accuracy scoring system development and testing
  • Week 4: Finalised product and performed end-to-end testing. Also explored Machine Learning integration

Week 1

Research Conducted

  • Worked on predicting note based on audio
  • Initial exploration of raw .WAV audio processing with the Fast Fourier Transform (FFT)
  • Extracted features for Neural Network training
  • Development of neural network approaches for pattern recognition

Week 1 Conclusion: I looked into different ways of being able to identified note played through raw audio files. After receiving feedback from my mentor, I was advised to move to MIDI. This was due to the short time-frame of the competition.


Week 2

Coming into week 2, I wanted to really focus on getting notes played on my keyboard displaying on my laptop.

Real-Time Processing

  • Stream processing algorithm for live performance analysis
  • Minimal latency feedback for immediate responses
  • Intuitive display of notes played on MuseScore

Week 2 Conclusion: I successfully created a process that allowed me to display what the student played in real time. I had problems with serial communication, but I fixed these issues to create a seamless response.


Week 3

At the half way point of the competition, I had laid the groundwork to allow me move on with the main part of the project: The accuracy scoring.

For this I designed an algorithm to discern whether the note played by the student was correct or wrong.

Accuracy Score

  • Notes detected were saved as .musicxml files using subprocesses
  • Notes pulled from score and recently saved file and difference was compared
  • Multiple rounds of note sanitisation carried out
  • Accuracy score displayed to student

Week 3 Conclusion: I made lots of progress this week, writing my accuracy algorithms and fixing data formatting issues. The largest one of these was converting the native Musescore files to MusicXML files. This allowed me to perform seamless correction of the student's playing.

Going into the last week, I knew I had solid ground to work off of, but I wanted to make improvements to the workflow and explore ML integration.


Week 4

Final Touches

  • Note-by-note accuracy assessment with confidence scoring
  • Displayed the correctly notes played back to the student on sheet music
  • Explored using Machine Learning to optimise the process

Week 4 Conclusion: This week polished off the project, focusing on the process itself and also ML integration. I submitted in my final report the working prototype as well as areas to improve if I were to continue working on the project.



Reflection on Project

There was lots of areas that I am happy with, and other areas I wish to improve. Considering that I went from ideation and planning, to building and implementing all just in 4 weeks, I am extremely proud of my project.

I am also very happy with the feedback I received from the TECS judges (and for awarding me first place!). They were very impressed with the project and the work that I put into it.

Here are some of my reflections on the project:

Ideation

  • I was very happy with the ideation process. I was able to come up with a clear plan of what I wanted to achieve and how I was going to do it
  • I spent lots of time conducting market research to see if there was a gap in the market for this type of product
  • I realised the importance of finding a real-world problem to solve, and then building a product to solve it
  • This gave me a purpose to build the project, and I was able to focus on the most important parts of the project

Planning

  • When I reflect upon my project, I realise how important planning actually is
  • I was able to come up with a clear plan of what I wanted to achieve and how I was going to do it
  • This allowed me to stay focused and manage my time effectively

Building

  • I was able to therefore prioritise the important parts of the project based on my planning
  • I built quickly, focusing on an iterative approach
  • I was able to get a working prototype in a short amount of time

Selling + Marketing

  • Though I never commercially sold this product, I did have to sell my work in the weekly reports
  • I focused less on the code itself, and more on the value that this solution gave to the user
  • This focus on value allowed me to tell the story of the problem, and how this solution solved it


Final Conclusion

The learnings that I have taken from TECS and applied to everything that I do is immense. I have been able to apply the same process to my other projects, and I have been able to get a lot more done in a lot less time.

The "learning by doing" aspect of this was incredible. I learned so much more by just getting stuck in.

This made the whole process more enjoyable, and I would recommend the competition to anyone who just loves building cool stuff.