Carlos has been working on WebGL since the beginning of the year. He’s really happy to be working with the Three.js engine. You can see his previous demo of WebGL online (the 2010 Ferrari F1 racing car) here: http://www.helloenjoy.com/helloracer.
Carlos and his team have just finished an interactive music video for Ellie Goulding, and he gave us a première. It’ll be up by the end of the week at lights.elliegoulding.com. The names in the animation are taken from twitter, as part of the interactive element
A quick Q&A:
Q. What happens when someone arrives in a non-WebGL browser (like IE)?
A. They get a screen saying, “Get a different browser”.
Q. Are the names clickable?
A. No, it takes people out of the experience
Q. Did you use the Audio API to create the undulating terrain?
A. No, they used FMod to hook up the audio. They chose which parts of the songs to put the spectrum to. Then put the sound into a JSON object, it was a 5Mb file, they put that in a PNG and then translated that.
Q. What was the hardest effect?
A. Rain. Up to 10 different renders in one frame.
Q. With so many rendering passes, how was the performance?
A. There were a lot of phases where it was terrible, they took stuff out to keep it working
Q. How long did it take to build?
A. Supposed to take 2 months, it took 1.
Q. Did you get any support from Ferrari or the F1 world for HelloRacer?
Next up, A 50-line useful WebGL app (Ilmari Heikkinen) with three.js
Ilmari works as a developer programme engineer. He’s on twitter as @ilmarihei
Three JS is a JavScript JS engine. It’s lightweight, efficient, it comes with fancy things that you can do. We’re focusing on WebGl but it has canvas and SVG render paths too.
He wants to show how to have “a hell of a time” with WebGL.
The Basic Setup
The first thing you need is a renderer. We clear it with opacity 1 and then clear the canvas. The result is a grey box on screen.
If you want to draw anything you need a camera, then you need a scene (this is the object that you are drawing), you draw the scene from the camera. Ilmari shows 10 lines of code that creates a scene with a cube. The cube is added to the scene as solid black. The result? He shows the same scene of a grey background with a black square in the middle, not quite interesting yet.
Can we make it move? He creates an animate function, it puts a camera at a position and moves it around in a circle. the code renders a scene for every frame. After drawing we keep moving. The result is a black cube that rotates using requestAnimationFrame() (like setTimeout but “friendly”, use it).
He shows code that creates a spotlight. He shows code that brings up a black cube with a lit square on top.
Three.JS has some shadows support. You need to enable them with the renderer. However, with just the cube this won’t be very interesting since the shadow will be cast on nothing. He creates a ground plane, and he makes the cube spin. The result is a cube that rolls around the ground plane. There’s even an amount of interactivting built it, you can move it around.
You can create a light from a point source. This is the start of shading effects and real-looking stuff. You can have up to 4 lights in a scene.
Now he gets technical (!). Shaders are what WebGL is really about. You can use the vertex Shader and the Fragment shader. You put them on the screen and tell it what pixels to draw. They’re written in GL Shading Language (GLSL). Vertex shaders allow you to manipulate triangles without modifying the mesh. Fragment shaders control the colour of points on the mesh.
He shows some code for a shader, quite complicated stuff but there’s lots of examples online. The vertex shader is first, then the the fragment shader. You can have multiple shaders, creating complex effects.
Useful uses for WebGL
Data visualisation. He shows code to create a bar graph. It looks through the values of a 2D array and creates a new cube from this. One slide of code gives most of the code necessary for a 3D, interactive, bar graph.
How to do text?
Create a canvas, draw some text, use it as a texture. Use “tex.needsUpdate = true;” <– do this or you’ll spend half an hour debugging it. He shows the result of the code on the screen, it’s a nice 3D “Hello World”. You could do this with CSS (but his talk’s about WebGL). You can even create video textures in the same way.
Creating GUI controls
Ilmari usually creates his own graphical user interfaces, but recently he’s been using DAT.GUI to simplify things. He shows the result of this code, it’s a scalable cube. It’s really simple to create a GUI control panel. But where is the pain? He shows code to change the current object position (this looks harder).
You’ve gotta have something that looks good to play with. He’s interested in the COLLADA Loader (Maya can export to this format). He uses that to load a monster. He adds the monster to a scene. The result after 15 lines of code if a moster walking across a ground plane.
All of this is made possible with github.com/mrdoob/three.js
Q. What about support for WebGL in Android?
A. It works in Firefox, but not very well.
Q. Is there a technique for multipass rendering?
A. You can clear the render manually, you can then call each render indivudally and layer them up.
Q. What’s the advantage of using Three.JS over WebGL directly?
A. Three.JS uses much less code. Something line 150 lines of code in Three.js is generally equivalent to 1000 lines in WebGL.
Q. Performance tips?
A. Don’t create too many meshes.
Q. Has Three.js been used in any big projects?
A. Yep, Three Dreams of Black, the Ro.me video
Q. Is there a way to put JPEG images on some of these planes?
A. Yes, On his text slide he shows how he uses a canvas, but he could’ve used an image, or even a video
Q. How does this respond to touch?
A. You have to add in all events yourself, including hover and mousedown events and the touch events.
Q. In your slides how do the camera
A. MouseMove event, he changed camera position using the Sine Cosine coordinates of the camera.
Q. Does the scene object support object graphs?
A. Yes, you can nest objects in the scene object and it’ll work.
Q. Where is the largest bottleneck for performance? Is it WebGL, Three.js,?
Q. What’s the most common usage for this library?
A. Doing fast programmable drawing. Fancy graphics effects. It does work for bar graphs, but Canvas rendering might actually be more compatible. However, things done in 3D can often be faster than in 2D
Q. How advanced could an interactive experience get? I’ve not seen any real physics yet.
Q. Can you have multiple cameras in a scene?
Q. Does Three.js on canvas offer the same abilities as WebGL?