Projection Mapping with Shadertoy and FFmpeg

WARNING! This workshops is a part of the event: "Mejker - ogólnopolskie spotkanie mejkerów". If you want to participate in the workshop, you need to register to the event as well: https://forms.gle/f5LbpM74L8LLY5Hn6

Description:

In this workshop, participants will learn how to compile and use the open-source multimedia processing software FFmpeg to enhance videos with unique filters using the GLSL shader repository Shadertoy (workshop part 01). Participants will then learn to use Raspberry Pi mini-computers with OFXPiMapper software to do projection mapping (workshop part 02).

FFmpeg is a powerful multimedia processing tool that can convert a video to any format. It can crop, scale, and apply filters, and it can do it really fast. It is free, open-source, and used by many commercial and free software products. It is a core part of the famous video player VLC, and YouTube, among many others, is using it to process videos.

Shadertoy is a community-driven shader playground that one can use to learn the GLSL shading language and share the source code. Shaders are used mainly in 3D graphics production (games, virtual, and web experiences); however, they can add value to video production.

Raspberry Pi is a low-cost mini-computer, developed in the UK to fill the market gap of affordable and hackable computers for kids. Since its release in 2012 it has become a computer that is used in a multitude of industries. It is a highly hackable and adaptable computer with exposed GPIO pins that invite users to connect sensors and actuators to it in order to achieve highly creative output.

OFXPiMapper is open-source software developed by the author of the workshop (Krisjanis Rijnieks) as his master's thesis at the Aalto University in Finland. It is an easy-to-use projection mapping solution that enables the use of the Raspberry Pi mini-computer for projection mapping. Projection mapping is texturing of real-life objects using a digital projector. Textures can be matched to the shape of an object by modifying the image in computer software real-time, before it is being sent to the projector at high frame rates.

Part 01, 11:30 - 13:10

Compiling and setting up FFmpeg with the vf_shadertoy filter on a Linux system. Applying Shadertoy filters to videos.

Part 02, 15:40 - 17:20

Setting up Raspberry Pi with OFXPiMapper software. Preparing videos and using them for projection mapping.

Warning: this is not a lot of time for a workshop like this, thus participants with no prior experience with Linux, command line interface and bash are expected to take the role of a spectator. If you have previous experience with Linux, command line interface and bash scripting, you are welcome to join with your system and follow along as fast as you can!

Prerequisites (Participants):

Equipment:

Participants are expected to bring their own laptop and should be able to install new software on their systems (administrator privilege).

Operating System:

Linux (Ubuntu or Debian) operating system is recommended for this workshop.

Experience:

Participants are expected to have basic understanding of producing their own video files. Basic knowledge of shell scripting is recommended.

About the Author:

Krisjanis Rijnieks (https://rijnieks.com) is a new media professional and fab lab expert currently working at the Aalto Fablab at Aalto University in Espoo, Finland. He is managing the fab lab and teaching as a Fab Academy instructor. As the result of his master's thesis at the Aalto Media Lab in 2016, an open-source projection mapping software OFXPiMapper was created. He maintains the Fab Academy electronics library for KiCad EDA after graduating from the Fab Academy in 2018. In the meantime, Krisjanis likes to get involved in international creative projects such as workshops, residencies, and exhibitions. He believes in open source tools as they offer more freedom of choice when it comes to individual professional activity.