MidiFly® is a gesture based MIDI controller. Instead of pressing physical buttons, the user generates MIDI messages by moving his or her hands in the air, intercepting a virtual grid of pads. By default, every virtual pad generates a note message, unless it's disabled. A disabled pad can either do nothing at all, or send a dynamics related message.

MidiFly® has 3 main screens: the Performance screen the Note Editor screen and the the Grid Editor screen.

Performance screen

This Performance Screen is where the MIDI instrument is actually played. It is composed of: a 3D view of the interaction grid a 2D view of the interaction grid a debug view of the depth camera output a MIDI setup menu, and an interaction menu.

a. MIDI menu

port name is the virtual MIDI port opened by MidiFly®: in most host software (synths, DAWs, etc.) you just need to look for a MIDI port with the same name.

channel is the MIDI channel used (if you're not sure, leave the default value of 1, or check the requirements of the host software you want to use).

program is the MIDI program used (if you're not sure, leave the default value of 0, or check the requirements of the host software you want to use).

save saves any changes to the MIDI configuration to file.

load loads the last saved changes to the MIDI configuration from file.

b. Interaction menu

The mirror toggle will mirror the grid.

The notes toggle, when disabled, will prevent note from being sent, even if the user is interacting with the grid.

The bend widget allows you to setup the dynamic bend control, using the standard MIDI bend message: in most synth software this automatically map to the bend control. Clicking on the widget iterates between 3 states.

The after widget allows you to setup the aftertouch control, using the standard MIDI aftertouch message: in most synth software this automatically maps to the aftertouch control. Clicking on the widget iterates between 3 states.

The CC1 and CC2 widgets allow you to send custom CC MIDI messages: in most synth software these messages can be mapped to custom controls. Refer to the manual of your chosen host software to map it correctly.

The velocity slider sets the amount of velocity applied to each pad interaction. On a regular keyboard, velocity is normally mapped to the strength used by the musician when pressing a given key; since there's not such a thing as strength in MidiFly®'s interaction paradigm, velocity here is a function of how quick you get from near the grid to into the grid. When your hand is near the grid a yellow block will appear in the 3D view, to give you a visual feedback: if you keep your hand at this distance for a moment before pushing through the grid, activating a pad, your velocity value will be lower than if you stab the pad quickly.

edit notes brings you to the Note Editor screen.

edit grid brings you to the Grid Editor screen.

 

c. Dynamics/Expressive messages

By default the virtual grid sends out notes. When you disable the note message for a specific pad, you have the possibility to use that pad to send dynamics messages instead, using the bend, after, CC1 or CC2 widgets.

Clicking on these widgets iterates between 3 states: disabled, meaning the message is not active at all. X, meaning the feature is mapped over the horizontal axis of the grid. Or Y, meaning the feature is mapped over the vertical axis of the grid.

As an example, let's imagine that you want to use the top row of your grid to control the bend and the last column on the right to control aftertouch, while still using all the other pads to send notes. First you would setup your notes, as described in 1.2. Then you would disable all the pads in the top row and the last column on the right. Then you would click on bend until it's mapped to X. Finally you would click on after until it's mapped to Y. With this setup, whenever your hand is activating any pad not belonging to the top row or the right column, you would be sending the notes you mapped. If you push your hand in the position of the first pad on the left of the top row, you would send a message setting the bend to the lowest value; if you push your hand in the position of the last pad on the right of the top row, you would send a message setting the bend to the highest value. If you move your hand along the pads on the top row, you will be moving the bend value accordingly. Similarly, if you move your hand along the right column, you will be moving the aftertouch accordingly.

The CC1 and CC2 settings also have a numeric parameter: this allow you to specify the CC number you want. Normally this is an arbitrary parameter you use to map a specifc CC to a chosen parameter of a host synth (please refer to MIDI specs and your specific software manual, in case of doubts).

Generally speaking, these messages are mapped on to an axis similarly to the way a fader operates, so mapping them on a row or column will make that row or column act as a touchless fader. If all notes are disabled and dynamics messages are mapped on to the whole grid, the grid will become similar to a touchless KaossPad. Finally, the user is free to experiment with any possible combination of notes and dynamics messages.

intro

performance screen

Note Editor screen

The Note Editor screen is where you configure the notes associated with each pad of your grid.

a. Setting notes individually

click on a pad to shift the note 1 semitone up 

shift+Click on a pad to shift the note 1 semitone down

X+Click on a pad to disable or enable the note

 

 

 

b. Setting note for the whole grid

oct up and oct down buttons will shift all notes on the grid one octave up or down

The scale widget allows you to apply a scale to the grid. The first parameter is the scale itself; use the arrow buttons on the sides to explore the available scales. The second parameter is the key note; use the arrow buttons on the sides to move between notes. After selecting the 2 parameters, click on the apply button to apply the selected scale to the grid.

 

c. Other widgets

reload notes reloads the last notes saved to file

save notes writes the current note configuration to file

back brings you back to the Performance screen

Grid Editor screen

The Grid Editor screen is where you configure the size and composition of your grid. Every change in this screen is previewed in the 3D view and applied to the real world grid, so that you can test your settings before saving.

sensor distance is the distance in mm between the lens of your sensor and the centre of your grid. (i.e.: a value of 1000 will set the centre of your grid at 1m from your camera).

pads is the size in mm of the side of each pad (i.e.: a value of 100 will get you 10cm pads).

grid size is the number of pads on each side of your grid (i.e.: a value of 6 will get you a 6x6 grid).

save grid saves any changes to file.

reload grid will load the last values you saved to file.

back brings you back to the Performance screen.

As an example, let's assume that you have your sensor on the floor, with the lens pointing up. Let's also assume that you want an 8x8 grid composed of 20cm pads: you want to set grid size to 8 and pads to 200. After doing this you test your new setup moving your hand over the sensor. Let's imagine that you want to play MidiFly® while standing and that, during your test, you discover the grid is a bit too low to be comfortable for you: all you have to do is adjust the "sensor distance slider until you like the final position. When you're happy with your settings, you press the save grid button.

intro

performance screen

Platforms

MidiFly® has been developed and tested on OSX.

From september 2020 also for Windows and Linux.

Sensors

Developed and tested with a MS Kinect.

From september 2020 also for IntelRealsense, Kinekt for Azure,

Kinect 2, Asus Xtion, Orbbec Astra, Mynt Eye and others.

FAQ

Ist Midifly Software oder Hardware?

MidiFly® ist eine Software. Es ist derzeit für Mac OSX programmiert, wird jedoch bis September 2020 auch für andere gängigere Plattformen optimiert. Es ist mit einer Infrarotkamera verbunden, die den Betrieb des virtuellen Gitters durch einfache Bewegung der Hände ermöglich.

In welchem ​​Kontext wird MidiFly® eingesetzt?

Die MidiFly®-Software (in der MIDI-Therapieversion) wurde bei Patienten mit kognitiven Problemen wie Autismus und anderen degenerativen Erkrankungen, aber auch bei Patienten mit Stress und Psychose getestet. Im Musiktherapieset steht der MidiFly® im Mittelpunkt. Der Patient und der Bediener können interagieren, indem sie mit der Bewegung der Hände auf das virtuelle Gitter einwirken. Das Gitter kann auf beiden Seiten verwendet werden, das sogenannte Spiegeln. Diese ermöglicht Benutzern, sich selbst zu spiegeln und einen fruchtbaren Klangdialog zu beginnen.

Wie viele Versionen von MidiFly® gibt es?

MidiFly® hat eine einzigartige Version, die in verschiedenen Kontexten verwendet werden kann, von Unterhaltung bis Sport, von Meditation bis Unterricht. Es ist mit der Basissoftware MidiFly® Sounds verbunden, einer Reihe von Soundvoreinstellungen, die speziell für MidiFly® erstellt wurden. Im Bereich der Neurorehabilitation ist MidiFly® Sounds für therapeutische Zwecke mit spezifischen Geräuschen optimiert, die von unseren Versuchslabors entwickelt wurden, und richtet sich an qualifizierte Neurorehabilitationsbediener, die mithilfe des Anwendungsprotokolls von MidiFly® in den von unserem Unternehmen organisierten Online-Kursen geschult wurden. Abhängig von der Pathologie bestimmen regelmäßig eine Reihe von Upgrades das Update mit neuen Voreinstellungen.

Was kostet MidiFly®?

MidiFly® ist kein Kostenfaktor, sondern eine Investition: Es ermöglicht einer Institution oder einem professionellen Anwender, sich auf einen neuen Zweig der Musiktherapie zu spezialisieren, der sich an Patienten mit kognitiven und …….nicht-kognitiven??? Problemen richtet. Die Applikation erlaubt Musiktherapeuten, sich in ihrem klinischen Kontext auf hohem technologischen Niveau zu positionieren.
Den Preis für jede der MidiFly®-Versionen finden Sie im Shop-Bereich dieser Website.

© 2020 by Stefano Ianne  plaza de l'Ayuntamiento 7 - Valencia (Espana) 

VAT Y6883792B info@midifly.com