GPU Programming using Metal 4: Part 1
Part 1: Vertex Drawing
I recently started reading through and learning to use the Metal 4 API. This story shows how to do some very simple drawing using the Metal 4 API. I plan to follow up with more stories like this—both to capture clearly and concisely my own understanding and learning and to help others who find themselves using the Metal 4 API.
Simplest Working Example: a Point
Starting with the Metal 4-based SwiftUI view I wrote about earlier here we’ll simplify it by removing the background color injected from the Timeline view and letting the MTKView fall back to its default value for clearColor:
// The App.
import SwiftUI
@main
struct MediumMetal4ViewApp: App {
var body: some Scene {
WindowGroup {
Metal4View()
}
}
}
// The Metal4View (probably in its own file).
import MetalKit
import SwiftUI
struct Metal4View: NSViewRepresentable {
func makeCoordinator() -> Metal4ViewCoordinator { Metal4ViewCoordinator() }
func makeNSView(context: Context) -> MTKView {
let view = MTKView()
view.delegate = context.coordinator
view.device = context.coordinator.device
return view
}
func updateNSView(_ view: MTKView, context: Context) {
print(view.clearColor)
view.needsDisplay = true
}
}
class Metal4ViewCoordinator: NSObject, MTKViewDelegate {
var device: MTLDevice
var commandQueue: MTL4CommandQueue
var commandBuffer: MTL4CommandBuffer
var allocator: MTL4CommandAllocator
override init() {
guard let d = MTLCreateSystemDefaultDevice(),
let queue = d.makeMTL4CommandQueue(),
let cmdBuffer = d.makeCommandBuffer(),
let alloc = d.makeCommandAllocator()
else {
fatalError("unable to create metal artifacts")
}
self.device = d
self.commandQueue = queue
self.commandBuffer = cmdBuffer
self.allocator = alloc
super.init()
}
func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {}
func draw(in view: MTKView) {
guard let drawable = view.currentDrawable else { return }
commandBuffer.beginCommandBuffer(allocator: allocator)
guard let descriptor = view.currentMTL4RenderPassDescriptor,
let encoder = commandBuffer.makeRenderCommandEncoder(
descriptor: descriptor)
else {
fatalError("unable to create encoder")
}
// Drawing goes before endEncoding() here.
encoder.endEncoding()
commandBuffer.endCommandBuffer()
commandQueue.waitForDrawable(drawable)
commandQueue.commit([commandBuffer])
commandQueue.signalDrawable(drawable)
drawable.present()
}
}If we run this it will tell us that the background color is black: MTLClearColor(red: 0.0, green: 0.0, blue: 0.0, alpha: 0.0). We can, of course assign a value of our choosing to the clearColor property, e.g., by adding tomakeNSView() the following: view.clearColor = .init(red: 0.75, green: 0.75, blue: 0.75, alpha: 1.0).
Drawing
So how do we draw into the view? Almost everything we need is already present. What remains is to make at lest one render pass and tell its render command encoder what is the graphics pipeline and what to draw with it:
- we used the
MTL4RenderPassDescriptorto create an instance of aMTL4RenderCommandEncoder, - the render command encoder will encode a render pass into its command buffer (an instance of
MTL4CommandBuffer) including the draw calls and configuration, - the
MTLRenderPipelineStaterepresents a specific configuration for the graphics-rendering pipeline, including which shaders it uses.
To draw (render) we will add some draw calls and the pipeline state to the encoder (we haven’t added any draw calls yet so no state was necessary).
The pipeline state is an instance of MTLRenderPipelineState (this name does not start with MTL4 because it is not a Metal 4-only type). At the minimum, it needs to specify the vertex and fragment shader functions and the pixel format. We’ll create it using a free function makePipelineState somewhere in our view file:
func makePipelineState(compiler: MTL4Compiler, library: MTLLibrary) throws -> MTLRenderPipelineState {
let pipelineDescriptor = MTL4RenderPipelineDescriptor()
// The vertex shader.
let vertexFDescriptor = MTL4LibraryFunctionDescriptor()
vertexFDescriptor.name = "vertex_f"
vertexFDescriptor.library = library
// The fragment shader.
let fragmentFDescriptor = MTL4LibraryFunctionDescriptor()
fragmentFDescriptor.name = "fragment_f"
fragmentFDescriptor.library = library
pipelineDescriptor.vertexFunctionDescriptor = vertexFDescriptor
pipelineDescriptor.fragmentFunctionDescriptor = fragmentFDescriptor
// The pixel format.
pipelineDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
return try compiler.makeRenderPipelineState(descriptor: pipelineDescriptor)
}Note that most of the work is in describing (via the descriptor) what the state is . Most importantly, the "vertex_f" and "fragment_f" names refer to shader functions to which we will come back later.
The makePipelineState function will be called on coordinator initialization and to call it we will also need the compiler and library arguments. We’ll stash the result into a pipelineState property of the Metal4ViewCoordinator, but we don’t need to store the compiler and library since—at least for this simple example—we won’t need them anymore. So the properties and init() method of the Metal4ViewCoordinator will look like this:
class Metal4ViewCoordinator: NSObject, MTKViewDelegate {
var device: MTLDevice
var commandQueue: MTL4CommandQueue
var commandBuffer: MTL4CommandBuffer
var allocator: MTL4CommandAllocator
var pipelineState: MTLRenderPipelineState
override init() {
guard let d = MTLCreateSystemDefaultDevice(),
let queue = d.makeMTL4CommandQueue(),
let cmdBuffer = d.makeCommandBuffer(),
let alloc = d.makeCommandAllocator(),
let lib = d.makeDefaultLibrary(),
let compiler = try? d.makeCompiler(descriptor: MTL4CompilerDescriptor()),
let state = try? makePipelineState(compiler: compiler, library: lib)
else {
fatalError("unable to create metal artifacts")
}
self.device = d
self.commandQueue = queue
self.commandBuffer = cmdBuffer
self.allocator = alloc
self.pipelineState = state
super.init()
}
...
// rest not shownHaving done that, we can tell the render command encoder about the pipeline in the draw() function as follows:
func draw(in view: MTKView) {
...
// Drawing goes before endEncoding() here.
encoder.setRenderPipelineState(self.pipelineState)
encoder.endEncoding()
...Build and run the program and you’ll notice a crash when trying to create the library object. This is because there are no *.metal files in our project (or other sources of what will be in this library). This is easy to fix by adding a Shaders.metal file to the project, e.g., by selecting a new file from the Metal template—the file any name with the *.metal extension and can have the minimal default content without any actual code as long as it is there. With this fix, however, the program crashes with a different error message: a failed assertion `Render Pipeline Descriptor Validation vertexFunction must not be nil. Why is that?
Shader Functions
In makePipelineState we told the pipeline that it is using a vertex and fragment shader functions named "vertex_f" and "fragment_f", respectively. However, we have not yet defined these functions. Let’s put the following in Shaders.metal:
struct Vertex {
float4 position [[position]];
};
vertex Vertex vertex_f() {
return {
.position = float4(0,0,0,1),
};
}
fragment float4 fragment_f() {
return float4(1, 0, 0 , 1);
}Note that the vertex shader vertex_f can return any struct as long as one of its fields is annotated with the [[position]] attribute—this is required and communicates the position where what is being drawn is placed (vertex or something else). The vertex shader returns a fixed position at the center of the coordinate space. The fragment shader returns a fixed red color for any pixel it is called for.
With these additions, the program no longer crashes since the requested functions "vertex_f" and "fragment_f" are found in the library. But nothing is drawn in the view either because there are no draw calls to the encoder yet so this GPU code does not get to execute.
Get Giik Web Developer’s stories in your inbox
Join Medium for free to get updates from this writer.
A MTL4RenderCommandEncoder has a few simple methods called drawPrimitives* so we’ll start with one of those, the drawPrimitives(primitiveType:vertexStart:vertexCount) method. More specifically, we’ll use it draw a single point. We will add the simplest of drawPrimitives(type:vertexStart:vertexCount) calls to the encoder:
func draw(in view: MTKView) {
...
// Drawing goes before endEncoding() here.
encoder.setRenderPipelineState(self.pipelineState)
encoder.drawPrimitives(primitiveType: .point, vertexStart: 0, vertexCount: 1) encoder.endEncoding()
encoder.endEncoding()
...I’ll let you read the documentation here but note what it tells us — that the current vertex id is passed to our shader with the [[vertex_id]] attribute. In our case, there will be a single call to vertex_f with id 0 so we can just ignore the attribute—note that the vertex_f function does not have the attributed parameter but we could also add it like so vertex Vertex vertex_f(uint id [[vertex_id]]) and then ignore it in the shader code.
Also, since we’re drawing point primitives, the documentation tells us that “…the vertex shader must provide [[point_size]], or the point size is undefined.” So to complete the vertex shader we need to modify it to satisfy these requirements. The Shaders.metal file will look like so:
struct Vertex {
float4 position [[position]];
float pointSize [[point_size]];
};
vertex Vertex vertex_f() {
return {
.position = float4(0,0,0,1),
.pointSize = 30,
};
}
fragment float4 fragment_f() {
return float4(1, 0, 0 , 1);
}And with this we finally get our beautiful point rendered at the center:
Note that since the vertex position is [[vertex_id]]-independent, we can emit more vertices but that won’t change the rendering. Go ahead and change the vertexCount: parameter of the drawPrimitives call to 10 or 20 and the drawing won’t change. We see a single point at the center of the Metal coordinate space which extends from -1 to 1 in all 3 dimensions (the space, not the point).
Shader Deep Dive
Now that the basics are working, we can spend a minute thinking about what’s going on here. The vertex shader is the first programmable (by us) stage of the Metal rendering pipeline, responsible for transforming input data into the final position of each vertex, while also passing on other necessary per-vertex data (like color or texture coordinates) to the subsequent stages. The input data can be an actual vertex position to be transformed into another position or—such as in our case—a vertex index which the shader can use to calculate the vertex position. Given the drawPrimitives call we’re using the vertex shader will be called once for each value from vertexStart through (vertexStart + vertexCount - 1) which in our case amounts to a single call.
The fragment shader is the second programmable stage in the Metal rendering pipeline, responsible for determining the final color and properties of a single pixel (or fragment). For now, without going into details, note that this shader will be called more than once since our point consists of more than one pixels and each pixel must have its color computed. These multiple calls may and will occur in parallel for many screen pixels which is what makes the GPU very fast.
This is important and it took me some time to realize fully so I’ll repeat it: a vertex is not the same as a pixel. The former represents an object in the scene being rendered and the latter an actual pixel on screen. In our current program, the vertex shader emitted a single vertex object with point size 30 which requires painting multiple pixels on screen (900 pixels, I suppose?). The rendering pipeline knows which are these pixels and calls the fragment shader for each and every one of them. The fragment shader simply returns the fixed color to apply to each of these pixels it is called for.
Making the Point Circular
The default and only publicly known (to me, at least) behavior of a point primitive is a square shape but what if we wanted the point to be a circle? We can make this happen by discarding pixels (not emitting the pixel color) for pixels beyond the area occupied by a circular point. We need a way to tell whether the coordinates of a pixel lie within the disk or not. If within, do as usual and emit the color. If not, drop the work altogether.
Dropping the work is achieved by calling discard_fragment() which compiles just fine despite apparently breaking the uniform control flow (there is a code path which does not return a value of the return type). We cannot have a fragment shader consisting of just discard_fragment() since it has no code path returning the color value. We can however, have the following to please the compiler—running it produces no point since all pixel fragments are discarded:
fragment float4 fragment_f() {
discard_fragment();
return float4(1, 0, 0, 1);
}Ok, so how do we decide which fragments (pixels) to discard? Turns out, we can add another shader argument with the [[point_coord]] attribute. The “Fragment Function Input Attributes” section of the Metal Shading Language specification tells us that this parameter is of type float2 and is “Two-dimensional coordinates, which range from 0.0 to 1.0 across a point primitive, specifying the location of the current fragment within the point primitive.” This is pretty useful and here is the new fragment shader:
fragment float4 fragment_f(float2 pointPos [[point_coord]]) {
if (length(pointPos - 0.5) > 0.5) discard_fragment();
return float4(1, 0, 0 , 1);
}Note the Metal coolness: pointPos is a float2 vector (tuple) so pointPos-0.5 is a component-wise vector subtraction giving us a vector from the center (0.5, 0.5) of the point primitive to the fragment/pixel. The length length(pointPos-0.5) of this vector is the distance from the center to the pixel. If this distance is greater than the circular point radius 0.5 we discard the fragment. Alternatively, distance(pointPos, 0.5) > 0.5 also works as the condition for discarding, that is, the distance from the center to the fragment.
The net result is this beautiful dot:
A Small Performance Tweak
Note that the length computation in the shader requires taking a square root to find the distance between two points. This is computationally expensive (relatively speaking, of course) but more importantly, we do not need it. We can get away with length_squared which, as the name implies, skips the square root and returns the squared distance:
fragment float4 fragment_f(float2 pointPos [[point_coord]]) {
if (length_squared(pointPos - 0.5) > 0.25) discard_fragment();
return float4(1, 0, 0 , 1);
}A Practice Exercise
To check your understanding you can try and change the point shape to half point, say only the upper half like so. In the process of doing so, you may come to realize an interesting fact about the coordinate system of the point (the one of the [[point_coord]]-attributed parameter). If you find a documentation of this behavior anywhere (I did not), please share the link in the comments.
That’s all for now. If you like the material go on to reading Part 2: More Vertices in this series. And have fun drawing with Metal 4.
