About SongSAT

"From Bands to a Band"

Millions of pixels are created every day. We believe that millions of new pieces of music can be generated to allow a wider audience to appreciate the vast treasure trove of satellite data we have available today.

SongSAT is a global award-winning tool to share the beauty of the world in different mediums, expressing the wonders of satellite imagery through audio. This allows the beauty of satellite imagery to communicate to an audience with visual impairments to enjoy the wonders of the world from above too, or to be used by musicians to aid with musical writing blocks.

Our team produced an algorithm, SongSAT, to convert five distinct geographical areas (Arctic, Grasslands, Forest, Coastal/Water areas, and Mountainous regions) into songs with distinct, recognizable musical patterns that play back using MIDI.

How SongSAT Works

Simplified Explanation

SongSAT determines the most common landcover type in a LandSAT image to determine what style of music it generates. For instance, if an image is mainly covered with agricultural fields and a small lake, SongSAT will translate the image into rural-themed music. This is achieved by translating each pixel's (the small squares that make up an image) digital number into notes. This process takes the remainder of the pixel's digital number (0-255) divided by the number of notes in a scale. Hence, if the theme uses a C major scale, it would divide the pixel value by 8 and take the remainder. If the remainder is 0, it would play a C. If the remainder is 1, it would play a D, and so forth.

These generated outputs are converted into playable sheet music, which is then brought into a music software that can play the music back.

We're proud to announce that this algorithm is 100% open source, and you can view the code at: http://github.com/mcvittal/SongSAT

How SongSAT Works

Advanced Explanation

The first thing SongSAT does is determine what the predominant land class the image is. To do this, we sample 400 points on the input image and extract the land class at these sample locations from the Global Land Cover Facility MODIS dataset. We also compare to see if any points fall in any mountainous regions, using a dataset provided by the United Nations, since the MODIS dataset does not have mountains as a land cover classification. If the majority of the points fall within mountainous pixels, it returns mountainous as the land cover classification, otherwise it will return the dominant land cover class fount from the MODIS dataset.

In order to have music that represents the landscape, a bit of human intervention was needed to guide the algorithm in the right direction. We are working to support all land cover types, and have successfully created five of the seven dominant land classes (Mountain, water, rural, forest, and arctic have been created, with desert and urban remaining to be created).

Each theme has a corresponding set of notes that it can play (The water theme uses a pentatonic scale, the mountain theme uses a combination of semitone clusters and a variation on the minor scale, and the other two use variations on the major scale), and each theme also has a set of rhythms for the two voices in the melody to use that are selected at random. SongSAT takes the modulus of each pixel value to simplify the 0-255 range of the pixel down to the number of notes in the selected scale, and then maps the new value, which is the degree of the scale, to the corresponding note in that scale. To make a pleasing accompaniment, the first note of the bar selected in the melody is made the note for the accompaniment for that bar, with the accompaniment switching octaves on each note change to add movement and variation to it. Forest however uses a sixteenth note arpeggiating baseline and mountain uses an aggressive syncopated rhythm instead of a simple octave switching to better match the mood of the area. All these values are written to a MIDI file as output. As well, to keep the melody line flowing and not disjointed, it ensures that no jumps larger than a perfect fifth occur (For example, if the interval is from C to A, it will choose to go down a major third instead of up a major sixth).

The MIDI file at the end was then brought into MuseScore, a free and open source notation software to select more appropriate instruments that fit the mood of the piece. The forest theme uses the default harp sound that ships with MuseScore, while the rest were instruments chosen from a third-party soundfont file to make the songs sound better and slightly less artificial.

Next Steps for SongSAT

If our team chooses to pursue improving SongSAT, the next thing to do would be to add the remaining land classification themes to allow for global coverage of SongSAT, and to add more rhythm options to the songs to make it feel a bit less monotonous and repetitive, and possibly add in multiple sections to the songs generated. After these key parts to it are improved, the next step after that would be to make the program more user-friendly and accessible so that anyone can install and run it, or make it into a proper web application that generates the music on the fly.

Product Video

Team Salinity

Interested in learning more or having a chat with the team? Using the music from SongSAT for a project? We'd love to get in touch! Shoot us an email at teamsalinity@gmail.com, and we'll get back to you shortly.

Alex McVittie

Graduated from the University of Waterloo's Faculty of Environment specializing in Geomatics. He now works full-time at SkyWatch Space Applications Inc. as a Platform Developer on the Image Processing Engineering team, designing cutting-edge remote sensing algorithms. In his spare time, he enjoys bicycle touring and writing music.

Colin TM

Expected to graduate from the University of Waterloo in August 2019. He is currently studying Geomatics as a major, with a minor in Computer Science. He has gained technical skills through working at the University of Waterloo (Mapping, Analysis, and Design Lab), York Region Planning department, and City of Toronto. He enjoys cooking and challenging himself in his free time.

Corina Kwong

Expected to graduate from the University of Waterloo in August 2019. She is currently pursuing Geomatics as a major, with a minor in Computer Science. During her studies, she has gained developer experience from working at CIBC, York Region, CGI, and Scotiabank. She hopes to pursuit a career in front-end development.

Janet Hu

Graduated with a Bachelor of Environmental Studies from the University of Waterloo. she has gained experience in the GIS field working for York Region, City of Toronto and Environment Canada. She is currently doing her masters in Physical Geography at the University of Waterloo.

"We're salty so you don't have to be."