Comparing the simplified line shaved many milliseconds from the total computation time. A bunch of sample gestures for each shape in each direction are stored and then called for comparison.Īs a user draws his line, the line is simplified, compared against the library samples, and then smoothed to create the final form. We used a library called Dollar Gesture Recognizer to achieve this. When the user draws a circle or triangle, the program recognizes the shape and applies a unique sound and effect. Once back to the main thread, the geometry of the line is updated with new data - ready to draw on the next animation frame! Gesture recognition While it’s animating, the co-ordinates are all flattened into a Float32Array which is passed back to the main thread at lightning speed thanks to transferable objects in WebWorkers. When a shape plays a sound, the points are scaled up and snap back to their original position with a spring physics calculation. Inside the WebWorker, we maintain the state of each shape in the form of a Vector object with X and Y co-ordinates for each point that is drawn as part of the shape. We handled this by moving the physics math off of the main thread into a WebWorker.
CHROME MUDIC LAB SERIES
For Kandinsky however, each line is stored as a separate mesh, made up of a series of points and attached to a WebGL scene.īelow you can see a wireframe example of the resulting mesh.Ī big challenge in calculating a large number of interactive objects at once is trying to render the animations consistently at 60fps on every supported device.
In regular drawing applications, the canvas is stored as pixel data and the pixels are manipulated for every stroke. Our final build uses WebGL to draw lines that are calculated and rendered on the GPU, resulting in a very smooth, dynamic and addictive experience. The second iteration used WebGL and turned lines into geometry by triangulation, which worked and was fast but gave us limited control over the look of the line. We started using HTML5 canvas to draw lines every frame, which was too slow. Check it out at Chrome Music Experiments. It was a collaboration with our friends at Google Creative Lab where we explored a bunch of different possibilities by sketching through code. This article describes the tech behind the experience. Some shapes even smile and sing back at you. Users doodle, scribble or draw on a web canvas and hit play to listen to their art work. If you’d like to share something with us, post it with #chromemusiclab or drop us a line.Kandinsky lets you make music through art. We always get inspired by new, unexpected ways that people use these experiments. You can play with these experiments across devices – phones, tablets, laptops – just by opening the site on a web browser such as Chrome.
CHROME MUDIC LAB CODE
You can get the open-source code to lots of these experiments here on Github. These tools make it easier for coders to build new interactive music experiences. Just open any experiment and start playing.Īll our experiments are all built with freely accessible web technology such as Web Audio API, WebMIDI, Tone.js, and more. Check out the Song Maker experiment, which lets you make and share your own songs. Here’s a collection of some uses we’ve found on Twitter. They’ve been combining it with dance and live instruments.
Many teachers have been using Chrome Music Lab as a tool in their classrooms to explore music and its connections to science, math, art, and more. Chrome Music Lab is a website that makes learning music more accessible through fun, hands-on experiments.
0 kommentar(er)
