Shader-up Your Game In Flutter
Initial look
Originally published on Pieces blog — https://code.pieces.app/blog/create-mind-blowing-visuals-using-shaders-in-flutter
Have you ever played a video game or watched a movie and been so blown away that you heard yourself saying, “WOW! I wonder how they did this?”
Most of the time, these rich visuals are created using a combination of shaders. Take a look at the visuals above, where shaders play a critical role. In this article, you’ll learn about a specific type of shader called a Fragment Shader and how to use it in your Flutter application. Outside of specifically discussing Fragment Shaders, I’ll touch on many other things including giving you some basic background on shaders for those who may not be as familiar.
By the end of the article, you’ll have learned about
- What are Shaders?
- How to write them.
- Creating your own shaders in Flutter.
With that said, let’s get started!🚀
1. Amazing World Of Shaders 🌏
Shaders are a set of instructions that a system executes for every single pixel on the screen. These are responsible for manipulating the color value, light and darkness for that pixel during the rendering/painting. Instead of painting layer by layer, you paint every single pixel on the screen, which gives you more control over your painting.
These instructions run for every pixel at the same time in parallel and thus shaders requires quite a lot of computational resources to run smoothly. Thus, shaders run on GPU(Graphical Processing Unit) which are especially made for handling tasks like this. Due to this, shaders are extremely fast.
2. Writing Shaders🧑🏽💻
Some of the languages for writing shaders are GLSL(OpenGL Shading Language), SKSL(Skia’s Shading Language), MetalSL(used by Apple devices), etc.
We’ll be using GLSL for writing our shaders. If you’re familiar with dart or c, you’ll pick it up easily.
Let’s look at a very simple example of a shader that outputs a beautiful gradient in GLSL, which will give you a basic idea of how to write GLSL code.
Preview :
Let’s go over the code step-by-step :
- Like Dart, GLSL defines a single entry point through the main() function. At the end of this function, we set the color for the pixel.
- GLSL has built-in variables like
gl_FragCoordandgl_FragColor.gl_FragCoordgives us the pixel vector position. And you set the pixel color through thegl_FragColor. When you’re specifying a vec4, you’re actually setting the rgba values of the color for that pixel. - GLSL allows creating conditional compilation blocks and defining values in the pre-compilation stage. These blocks or values start with #. #ifdef(if defined)- #endif is a conditional operation block that is checked before compilation. 2nd line executes if the GL_ES is defined, which is a version of GLSL for embedded systems like mobiles and video game consoles.
- The level of precision is everything when you’re dealing with colors. The more precise you’re, the better quality you get. The 2nd line sets the float precision to mediump. You can also change it to lowp or highp if wanted.
- Inputs for shader are defined by adding a uniform in front of them at the top of your code. One of the most important inputs is the u_resolution(also called as iResolution) of type vec2 which gives us the resolution of the screen.
If you want to learn more about shaders and how to write them in detail, check out thebookofshaders and shadertoy for examples.
3. Shaders In Flutter 💙
With the release of Flutter 3.0, initial support for creating custom shaders was moved to stable. So, make sure you’re running the latest version.
You can upgrade to the latest version through the following command.
> flutter upgradeIf you’ve been around Flutter for a while then you may already know that the Skia is the Flutter rendering engine that it uses for rendering UI. And the SKSL is the Shading language of Skia.
Flutter supports creating custom shaders through SPIR-V Dart API that was moved to stable in the Flutter 3.0. SPIR-V is an intermediary language designed to act as a bridge between different Shading languages for making it easier to port shaders from one language to another.
To use custom shaders in Flutter, you’ll be converting your GLSL code to SPIR-V binaries. For this, Flutter does provide a way which is mentioned here.
Once you got the binaries, you need to use something called FragmentProgram which helps compile the SPIR-V binary to SKSL and to create a shader from it. But, this is easier said than done!😬
This manual process is quite time-consuming and not that intuitive for someone who would like to just get started quickly with shaders in Flutter.
So, we’ll be taking look at a tool that will automate this process and generate the SPIR-V binaries from our GLSL code and also provide an intuitive API for using those shaders in Flutter.
4. Umbra
To make our life a little easier, we have an amazing tool from the community called Umbra that handles shaders compilation and creation for Flutter.
Umbra is a CLI and visual editor (WIP) for writing shaders in Flutter. It comes with an abstraction on top of the Flutter API to make the lives of developers that want to write shaders easier. The main goal therefore is to make writing shaders in Flutter easier, faster and overall more enjoyable — Umbra Docs
Umbra CLI takes care of the compilation of GLSL to SPIR-V and generates Dart API. The API handles the compilation of SPIR-V to SKSL and provides an intuitive strongly typed support for using your Shader.
It also adds some abstraction on top of your GLSL code which helps you focus only on writing your shader code and it adds the necessary raw GLSL code before it compiles the shader. You’ll learn about the abstraction Umbra adds in the next section.
Installing Umbra CLI :
- Activate the Umbra CLI by typing the following command in your terminal
> dart pub global activate umbra_cli2. Once activated, download the Umbra dependencies through the following command.
> umbra install-depsThis might take a while, so sit back and relax a bit!😴
5. Creating Your First Shader In Flutter ✨
You’ll learn how to add this cool shader below which kinda resembles the ripples in water but is a bit fancier :)
- Add the umbra_flutter package in your pubspec.yaml
umbra_flutter: ^0.1.0-dev.1or add through CLI with following command
> flutter pub add umbra_flutter2. Create a new shader project through Umbra CLI by typing this command in terminal at the root of your project.
> umbra create hello_worldThis will generate a GLSL file at the root of your project where you can add your GLSL code.
2. Add the following GLSL code to the hello_world.glsl file.
As you can see, some of the configuration things we saw in the earlier example of the GLSL code are abstracted by Umbra.
The main() function is replaced by a fragment function which gives us the uv(normalized coordinates of the pixel) and fragCoord(coordinates of the pixel without normalization-gl_FragCoord). And returns the pixel color instead of assigning it to gl_FragColor.
We also added another input called u_time , which represents the time since we started. It’s crucial when you want to animate your shader with respect to time.
3. Create a folder named shaders in your lib folder. Now, run the following command which takes the path of the GLSL shader file and the output location.
> umbra generate hello_world.glsl --output lib/shaders/This generates Dart files and the SPIR-V binaries at the given output location for your Shader.
Umbra now supports generating the Flutter widget which loads the shader. So, the following step may not be required for you. — checkout umbra
4. Create a my_shader.dart file somewhere in your lib folder. Add the following code within that file and import your HelloWorld shader in this file.
What we are doing here in short is that we first compile our shader and as the process is async, we use FutureBuilder to load it. Once it’s compiled, we then use .shader() method on our UmbraShader object and pass it required inputs.
The resolution would be the Size you want for your shader and for the uTime, we are passing it the delta (time since we started). It’s calculated with the help of Ticker. Ticker triggers a callback for every frame which increments the delta by 1/60 ms.
Finally, we pass the created shader to the CustomPainter to paint.
If you’re going to create shaders often then I would recommend saving the above code snippet somewhere and just update the Shader type and inputs.
Build and run :
Awesome!🥳🙌
You just created your first shader in Flutter! Isn’t that exciting!
Src code for the project can be found here: Flutter Shaders Hello World
6. Limitations 🚧
There are a few limitations and constraints when writing your GLSL code for Flutter. Some of the limitations are from Skia, and some related to the current integration of shaders in Flutter itself.
A few of those are :
forloops must initialize a float variable to a constant value, compare it against a constant value, and increment/modify it by a constant value.- Doesn’t support
whileandswitchstatements. - The types that are supported are sampler2D, bool, float, float-vector types, and square float-matrix types.
- Only built-in functions present in GLSL ES 100 are used.
These are some of the rules mentioned in the SPIR-V library that you have to adhere to while writing your GLSL code. Some of these rules are handled by Umbra while generating the raw GLSL code but for some of them, you should be on the lookout.
Also, note that there’s no Flutter web support for shaders at this point.
These limitations will likely go away in future versions of the library.
7. What’s Next? 🙇♂️
This initial release for supporting Shaders in Flutter paves the way for some creative coding and unlocks new and never before seen experiences in Flutter. Though the Shaders support is still WIP and is not yet marked for production, we can see full support coming along the way.
While I was doing some research into shaders in Flutter, I found some interesting stuff on Flutter GitHub that may land soon in near future.
Some of these things adds support for compiling your GLSL shaders to SPIR-V through the Flutter CLI 💯
And SKRunTimeEffect. ✨
This would make it possible to create custom Shaders and ColorFilters with SKSL which could be pretty useful for image/video editing apps and games in general.
I hope you’re just as excited as I’m right now with what’s coming soon for shader support in Flutter.
Hope you enjoyed your time reading this overview of Shaders in Flutter. And thanx for spending your precious time reading the article.
You can follow me on Twitter @TakRutvik, where I share such interesting things now n then.
More on Shaders :
If this article got you excited about creating some cool shaders, then check out the ShaderToy for some amazing examples of what people are doing with Shaders and try to add those in Flutter.🙌
To learn about Shaders in detail, check out thebookofshaders.
