home | portfolio | publications


YPX – A Visual Programming Environment (2020 –)
YPX

YPX is a powerful multi-purpose data processing tool using a visual programming paradigm for configuring filter graphs and pipelines. It is meant as a rapid-prototyping tool for interactive experiences and spaces, for testing and studying user interface technology and devices, as well as their potential combinations.

YPX

Starting off as a tiny custom config-file based command line tool for presentation of our RESi yarn (hence, “Yarn Processing”), it was further and further extended and complemented with additional functionality and a powerful UI, as well as numerous data sources and sinks, to ultimately become Yarn Processing eXtended.

YPX YPX YPX YPX 
YPX is meant to be a tool for fast prototyping (i.e., setting up and tuning) of small-scale to rather complex data processing methods. The GUI is heavily inspired by well-known and widespread patching tools such as Max/MSP, Puredata, vvvv, TouchDesigner, etc., the user can easily set up directed graphs by instantiating filter nodes (“Operators”) and by configuring respective parameters and instantly see the effects at integrated live data visualizations. Existing Operators include basic algebraic operations, mathematic operations such as FFT, low- and high-pass filters, matrix rearrangements, color space conversions, logical operations, and much more.

Configurations (“scenes”) are saved to human-readable JSON files. Once configured using the GUI and ready to use, the scenes can be again loaded by an YPX instance, which also can be run headless, i.e., without UI as a console application, to save computing resources and visual clutter. To not depend on graphics interfaces (i.e., OpenGL), it can be compiled completely without UI-support using a preprocessor switch, which makes it feasible for, e.g., running a concluded processing stage on a shell-only edge-computing-device.

YPX YPX YPX 
The idea is to have YPX in between of raw data providers, such as HIDs or other sensing devices, and target applications/frontends (e.g., a C++, Unity3D, or Unreal Engine application on a PC, an Android or iOS app on a mobile device, a vehicle’s dashboard, or even another tool such as MATLAB for further processing). It would draw unprocessed data from the device, filter, and/or extract high-level information (e.g., input gestures), which will be sent to the application. To be as versatile as possible and to enable distributing components to several computing devices, it heavily utilizes networking interfaces (we prefer OSC via UDP for many use cases requiring interactive rate). Basically, any device that can send/receive data via network can be a sender/receiver. As OSC is a widespread protocol in the field of multimedia, it is easy to link YPX functionality to many existing tools (vvvv, Open Stage Control, MadMapper, etc.).

YPX YPX YPX 
This network-centric architecture also allows for prototyping applications on mobile devices by sending data via WIFI or Bluetooth Serial, or to implement a bypass or feedback loop, e.g., by streaming data to a Python script via TCP, for processing steps requiring SciPy, then (if required) streaming the result back to YPX for capturing, etc. Data can furthermore be captured to file (CSV or binary formats) and replayed (e.g., for batch-scripted testing and controlled evaluation of different processing methods from captured input).

The tool’s source code is written in C++ and built on-top of widespread cross-platform libraries uses to ensure portability at least among Windows/macOS/Linux. Visualization is based on OpenGL, data for processing it builds upon OpenCV, so a switch to GPGPU processing is easy to do (although not implemented at this point since never required). It can easily be extended by additional processing Operators, data sources, and sinks, for example adding VRPN functionality is straightforward. The UI is built with extensibility in mind; live data visualizers are replaceable so custom implementations can be added, e.g., replacing the line graph or heatmap visualizations of an Operator type by a 3D scene rendering can be accomplished easily.

YPX YPX 
For our custom made sensor hardware, we implemented a specific protocol that we usually transmit via serial, Wifi, or Bluetooth. However, any data can basically be used quite easily, as additional data sources are straightforward to implement. So far, the tool supports the following input devices and protocols for input and/or output.
Furthermore, streamed data can be compressed to save bandwidth using JPEG for image data, as well as the lossless methods MiniLZO, QuickLZ, bzip2, zStd, zLib, and LZ4 for generic data.

YPX YPX YPX 
We complement the code with sample implementations for senders/receivers: Platform IO based firmware for Arduino compatible MCUs, target application templates for Unity3D, Python, and Processing, as well as plugins for vvvv, MATLAB (mex-Plugin), and Grasshopper3D/Rhino.

YPX is currently in development (more precisely in a refactoring state for better modularity by providing a plugin-system) and will be open sourced on Github as soon as this overhaul is finished. In case you find the project interesting and want to learn more and/or contribute, feel free to get in touch.

Context: Multi-purpose data processing and streaming environment created for requirements of TextileUX and beyond
Contribution: Design and development of a solution for a data processing pipeline for multi-purpose data processing, analysis, and streaming 
Credits: Media Interaction Lab, University of Applied Sciences Upper Austria


© copyright 2024 | rolandaigner.com | all rights reserved