REALTIME WEB TECHNOLOGIES IN THE NETWORKED PERFORMANCE ENVIRONMENT

- http://rob.kiben.net rob@kiben.net -

Rob Canning
Department of Music
Goldsmiths, University of London

NODESCORE

NodeScore is a web based framework to facilitate networked ensemble performance.



  • NOT a synthesis system
  • NOT a composition tool
  • NO control level output (OSC etc.)

  • DOES push cues/notation to human performers
  • Precomposed or on-the-fly

LOCAL


  • Telepresence vs. Presence

    • "Hypertextual"/ NonLinear Scores/Strategies
    • Composed/Mediated Improvisation
    • London Improvisors Orchestra (LIO) "Conductions"
    • Zorn's Cobra - Flash Card Rule based Systems

    Dramaturgies

    Interaction - Distributed Controls/Authorship
    projected, directed and distributed
    BreakDOM game screenshot BreakDOM game screenshot

    Performance Problems


    • Clutter - clicktrack, stopwatch, metronome
    • Oversized/complex "hypertextual" paper scores

    • Different Demands to the Laptop Orchestra
    • Rich Media v. Composition tool + plugin

    NODESCORE


    • Scores assembled in HTML5
    • Musicians displays controled remotely

    • Not all-in-one solution - Modular Approach
    • May run alongside other streams: media, OSC automation etc.

    Standards Compliant

    BreakDOM game screenshot
    • HTML5
    • CSS3
    • (JAVASCRIPT)
    • WEBSOCKETS


    Open Source - No Plugins
    (Flash, Flex, Shockwave, Java)

    traditional model: http req res

    Server Push Technology

    • AJAX/Comet Programming/Long Polling
    • FLASH / JAVA - FLEX - BLADEDS

    • BOSH
    • WEBSOCKETS

    Websockets

    BreakDOM game screenshot
    • More efficient than AJAX/Comet Programming
    • Persistant bidirectional TCP socket
    • REQUEST/RESPONSE handshakes elimintated *

    HTTP://SOCKET.IO

    Socket.IO simplifies the WebSocket API and unifies the APIs of its fallback transports.

    BreakDOM game screenshot

    Directed

    BreakDOM game screenshot

    Distributed

    BreakDOM game screenshot

    Serverside Javascript

    NodeJS Server

    var sio = require('socket.io')
    , http = require('http'), fs = require('fs'), static = require('node-static');
    
    var clientFiles = new static.Server('./www');
    var httpServer = http.createServer(
        function(request, response) {
    	request.addListener('end', function () {
    	    clientFiles.serve(request, response);
    	    process.setMaxListeners(0);
    	});    
        }); httpServer.listen(8889);
    
    var io = sio.listen(httpServer);
    io.set('log level', 1); // reduce logging                                                                
    io.sockets.on('connection', function (socket) {

    Client Plumbing

    // load the sockets lib
    <script src="/socket.io/socket.io.js"> </script>
    		
    // connect to socket
    
    var socket = io.connect();
    
    //send something down the pipe
    
    function startChr() { socket.emit("startChr") }                                                          

    CLIENT->SERVER->CLIENT

    Chronometer on server pushes -> client
    // instruction from "director" client executed on server
    socket.on('startChr', function () { startChr();}); 
    function startChr() { if (chronstate !== 1) { 
       chronstate = 1; chronometer();} }
    	    ---snip--- 
    	    setTimeout(function(){chronometer()}, 100);
    	    socket.broadcast.emit('chronFromServer', chron)
    	    socket.emit('chronFromServer', chron) 
    	    ---snip---
    // update chron value on the clients in line with server
    socket.on("chronFromServer", function(chron) {
        $("div#client_chronometer").text(chron);
    }); 

    NODESCORE: INFO PANEL

    screenshot

    NODESCORE: CHAT PANEL

    screenshot

    NODESCORE: Score

    screenshot

    GNU/Lilypond - Phantom.js

    Server Sequencer

    DEMO

    Future Work

    the TODO list...

    1. Further Modularisation
    2. Server on embedded linux (Gumstix) + openwrt
    3. Animated SVG, WebGL, <audio>,<video>
    4. Annotation via stylus?
    5. work with an ensemble over extended period - eg. residency
    6. touch screen "director" interface
    7. clean up the code :)

    links

    This Presentation: http://nodescore.kiben.net/p/

    http://nodescore.kiben.net

    ICMC2012 Paper

    src: http://gitorious.org/nodescore

    rob@kiben.net

    View Rob Canning's profile on LinkedIn