Originally created to forward UDP TUIO (OSC) messages straight to TCP to be read from within Flash.
This application forwards all incoming UDP messages straight to TCP without touching the data, just a straight forward.(Since version 0.2.1 there is the option to prefix the size of the packet before sending the data to comply with OSC / TCP specifications). This enables applications that don't support UDP (e.g. Flash) to receive the data. Since OSC / TUIO are generally sent via UDP, this enables Flash to recieve those messages in their raw binary form.
So my MSA Remote app has been rejected AGAIN, for the second time! This time because my startup image "infringes an Apple Trademark Image". Aaargh!
I had paid special attention to not include the sliver outline and home button in case it would be an issue, but it wasn't good enough. Apparently Apple own rectangles with rounded corners at a specific radius.
But like they say, Never give up, never surrender. I've resubmitted with a much tighter crop (I know, a lot uglier :S but it'll have to do for now), if this doesn't get approved, I think I'll just give it all up and retire to a little fishing village (maybe I should do that anyway).
P.S. You can see the previous post about the first rejection here.
Demo, Velocity Sensitive Keys and Patents
In the meantime I've uploaded a new video, this time demoing controlling Ableton Live (for audio) and VDMX (for visuals) and using faders, triggers, and velocity sensitive keys - yes velocity sensitive. The harder you hit the keys, the louder the sound (or whatever you want to map it to).
Coincidentally, in a tweet from cdmblogs I saw these guys have developed a very similar "patent pending" technique. They're very young, all in their early twenty's. So respect guys for all you've developed and the whole business venture etc. but a tip, please don't try and patent things like this. It's bad for mankind.
And for those wondering how it works, it's not rocket science, it's accelerometer. The harder you hit the screen, the bigger jolt on the accelerometer. And yes it does work on hard surfaces because you still get an internal jolt. You just need to do some filtering and check change of acceleration on finger touchDown. Neat I know ;) patent worthy? I think not.
In this video, OSCulator is routing the OSC (& TUIO) messages coming from MSA Remote to midi and forwarding to Ableton Live and VDMX simultaneously. Nothing is done in post, the same signal is controlling both audio and video.
MSA Remote is a remote control application for iPhone & iPod Touch that sends OSC messages over the wifi network. This allows you to control any OSC supporting applications such as Max/MSP/Jitter, PureData, Reaktor, VDMX, vvvv, Resolume, Quartz Composer etc. as well as any custom applications such as interactive installations, audio/visual performances, multitouch capable surfaces etc.
By mapping the OSC to midi on desktop (e.g. using OSCulator) allows further control of any application which supports midi such as Ableton Live, Cubase, Logic Pro, 3DSMax etc. In addition, developers can easily integrate OSC into their applications knowing it can be controlled remotely.
The application can be distributed to visitors, guests, members of the public etc. to interact with an installation or performance, or used by dedicated performers.
- Multitouch information sent using standard TUIO protocol for instant integration with existing TUIO clients
- Accelerometer data for each axis (x, y, z) is sent
- 64 faders (8 pages of 8 faders)
- 64 triggers (8 pages of 8 triggers)
- 108 key (9 octaves) VELOCITY SENSITIVE polyphonic keyboard. Yes, the harder you hit the keys, the greater the velocity.
- Settings are automatically saved and restored
- Multitouch area orientation can be set as desired
- All information on protocols are documented in the app
Controlling VDMX & Ableton Live simultaneously
In this video, OSCulator is routing the OSC (& TUIO) messages coming from MSA Remote to midi and forwarding to Ableton Live (for audio) and VDMX (for visuals) simultaneously. Nothing is done in post, the same signal is controlling both audio and video. Video demonstrates faders, triggers, and velocity sensitive keys - yes velocity sensitive. The harder you hit the keys, the louder the sound (or whatever you want to map it to).
Controlling Java/processing.org fluid simulation with TUIO
Source code for the processing.org sketch can be found here:
Controlling custom multitouch paint application with TUIO
Graffiti Wall is a product that Tangible Interaction had already developed a while ago and have been using commercially. It was built on TUIO from the beginning, so when I passed on a copy of MSA Remote (via ad-hoc), they were able to immediately integrate it with their app without any modifications.
Camille Troillard, the mastermind behind OSCulator was kind enough to add some features (and sample patches) to his app to make handling the data coming from MSA Remote and converting to midi as seamless as possible.
The /msaremote/fader protocol is recognized and faders are automatically setup to forward to midiCC with same number as the slider.
The /msaremote/trigger protocol is recognized and triggers are automatically setup to forward to midi notes with constant velocity.
The /msaremote/bank protocol is recognized and triggers are automatically setup to forward to midi notes with constant velocity.
In the Osculator/Sample Patches/iPhone folder there are two sample patches which map incoming msaremote keyboard OSC messages to midi notes with velocity. The basic patch maps each key to the relevant midi note, while the advanced patch has the keys setup so you can assign each key to a different event.
Max/MSP
Blair Neal aka vimeo.com/laserpilot has created a Max MSP template (for Max5) to handle the OSC Messages coming from MSA Remote. Cheers Blair!
This is a library for solving real-time fluid dynamics simulations based on Navier-Stokes equations and Jos Stam's paper on Real-Time Fluid Dynamics for Games. While I wrote the library primarily for processing it has no dependency on processing libraries and the source can be used with any Java application.
The video below is a demo of a processing sketch using MSAFluid, being controlled by MSA Remote on iPhone (You can view the video in HD and download a 1080p version at vimeo).
This is a tutorial in getting a very simple openFrameworks application running on iPhone with basic graphics, multitouch and accelerometer support (and one might say a simple particle system too!).
10 Balls are moving around on screen and bouncing off the edges.
You can touch the screen with multiple fingers and drag the balls around (multitouch support)
You can tilt the iphone and the balls fall in that direction (accelerometer support).
...and all of this without touching a line of Objective C. It is actually one of the samples included in the ofxiPhone download - iPhone Touch+Accel Example. You can find it in the examples folder of the download, so if you load and run that project you can see the finished result. The code below is straight from that sample, warts and all :P
An interactive, generative, audio-reactive visualization app for the iPhone and iPod Touch (what a mouthful!) (This is the continuation of what I originally called "The Meshulator").
Use your fingers to create and interact with abstract geometry in 2D or 3D. Very simple to use, just slide your fingers over the screen and try different gestures. Tilt the device to look at the shape from different angles. Options for two different kinds of audio-reactive behaviour (microphone input - iPhone only): deforming based on audio, or completely generative driven by audio. If you feel adventurous, you can play with plenty of customizable parameters to adjust coloring, physics, shape etc. Or if that feels too intimidating just click 'randomize' to make the settings random and experiment.
It was written with a custom version of openFrameworks (pre-006) and an early version of the ofxiPhone addon. Information on openFrameworks and ofxiPhone at memo.tv/ofxiphone.
148apps.com - 4/5 stars."You will find yourself showing people this application …and then, not getting your iPhone back because they want to keep painting."
geek.com - "I like the fact that the app is not just a port of the website, but includes enhancements which take advantage of the features of the iPhone."
iphonefreakz.com - "...the most jollificating paint app for iPhone and iPod Touch. It’s like getting a Jackson Pollock soul inside you...Definitely worth having it on your phone for. Well worth $0.99"
iPhone adaptation of Miltos Manetas' website jacksonpollock.org (original flash developed by Stamen Design). Save your pictures and send them to jackson@jacksonpollock.org, the best Pollocks will be published in an upcoming book.
The video below shows the basic features of v1.0. In the current version (v1.1) ability to choose your own colors has been added to allow the creation of more controlled paintings like those seen in the images below.
Version 1.1 is now in the appStore. New version brings serious optimizations thus max amount of dust is now tripled.
This is a port of my original gold dust app (vimeo.com/2281077) to iPhone. Of course it doesn't translate exactly but this is my first attempt to capture the essence of the original demo. Unfortunately I don't think this video manages to capture the behavior of the glittery gold dust either, but it's free so you can try it for yourself :P.
It was written with a custom version of openFrameworks (pre-006) and an early version of the ofxiPhone addon. Information on openFrameworks and ofxiPhone at memo.tv/ofxiphone.
Well I finally caved in and bought an iPhone - and my favorite feature (and main reason for buying it) is of course the multi-touch capabilities. So currently OSCemote is my favourite app. Apart from having a few sliders and knobs which transmit OSC (similar to TouchOSC), it also has a multitouch pad which sends out TUIO messages, so any app which responds to TUIO (E.g. anything written with reactivision api) will respond. So I had to try out my visualisation for the Roots Project! Up and running in 5 minutes! Awesome! (I had to rotate the coordinates in my processing code though to have the long end of the iphone screen map to the long end of my desktop screen, slightly annoying that this isn't an option in the app... hopefully soon :P).
"Roots" is an interactive musical/visual installation for the Brick Table tangible and multi-touch interface, where multiple people can collaborate in making generative music in a dynamic & visually responsive environment. It is a collaborative effort between myself and the Brick Table creators Jordan Hochenbaum & Owen Vallis. It will premiere at the Minitek Music + Innovation Festival September 12-14, 2008 in New York.
The essence of the interaction, is that you control parameters of a chaotic environment - which affect the behaviour of its inhabitants - which create and control music.
To breakdown very briefly without going into much detail:
There are vinelike structures branching and wandering around on the table. They live and move in an environment governed by chaos.
Audio is triggered and controlled entirely by how and where the branches move.
You - the user - control various parameters of the chaotic environment. Parameters which range from introducing varying amounts of order, to simply changing certain properties to let the chaos evolve in different directions.
There are varying levels of interaction, ranging from traditional one-to-one correlations - 'this movement I make creates that sound', but also to more complex relationships along the lines of 'this movement I make affects the environment in this way which sends the music into that direction where it evolves with a life of its own'. The visuals are purely generative, as is the audio, and as user you can play with the parameters of that system and watch and listen to the results...
Demo of drawing with roots:
Demo of using fiducials to create magnetic force fields:
This is a demo of creating and visualizing magnetic (kind of) fields in Processing and controlling with a tangible multitouch table and Quartz Composer. It gets more interesting after the 1 minute mark :P
Thanks to the windy ways of the web, I've found myself working with some truly talented musicians/techies/electronics experts over on the otherside of the pond in California, on a very exciting interactive, generative audio/visual project. The number of traditional instruments they have and play wasn't enough for them, so they decided to build their own, as one does when in that situation - one of which is the bricktable, a tangible and multi-touch table - and instrument.
I've worked on a number of interesting interactive audio projects, but the approach in this one is quite different and i"m very excitied to be working with the bricktable guys on it.
In one line: You control parameters of a chaotic environment - which affect the behaviour of its inhabitants - which create and control music.
To breakdown very briefly without going into much detail:
Aldeburgh Music is an organization based in Suffolk, UK working with musicians - both professional and just starting out - to help them reach their full potential by providing them with the time and space to discover, create and explore - as well as providing inspirational scenery and a rich musical heritage.
The New Music New Media / Britten–Pears Programme offers advanced performance experience to young professional musicians in the inspiring surroundings of Snape Maltings, home of the Aldeburgh Festival founded by Benjamin Britten in 1948.
