Contents

• Introduction
• Optimizations
• Final remarks
• References

This sample uses OpenGL ES 3.1 and the Android extension pack to perform displacement mapping with tessellation. The sample investigates common techniques used to improve performance and visuals.

Introduction

The Android extension pack adds many new features to mobile, that have until now been exclusive to desktop OpenGL. New pipeline stages, such as geometry and tessellation shaders, are now available to the programmer. This sample showcases the use of tessellation on a Mali GPU.

In the sample we apply a displacement map to a coarse mesh, to generate detailed geometry on the fly. We combine this with a screen-space based LOD computation to progressively vary the level of detail, depending on how much screen area the geometry covers.

About tessellation

Tessellation introduces three optional pipeline stages – two of them programmable – located conveniently after the vertex shader and before the fragment shader. These stages are capable of generating additional geometry. How many triangles, or where the generated geometry should be placed, is programmable in the Tessellation Control and Tessellation Evaluation shaders. These both operate on a per-vertex basis, and can see all the vertices of a single primitive.

The ability to generate additional geometry gives increased expressive power to the programmer. However, the ease of use is debatable. Realtime tessellation is notorious for requiring great care to avoid performance issues, patch gaps, mesh swimming or other visual artifacts. In this sample we will take a look at one particular usage of tessellation, as well as some tricks that can be applied to improve performance and hide artifacts. Before moving on, we briefly mention some common use cases, some of which we leave to the reader for further investigation:

Continuous level-of-detail (LOD): Geometry that covers only a handful of pixels clearly has lower requirements for detail than geometry that meets the viewer face-on. Traditionally this has been handled by dividing a mesh into several distinct meshes, each with a different level of detail. While this does the job, it is often difficult to hide the popping that occurs, when switching between discrete levels. Tessellation can be used to provide a seemingly continuous transition between detail levels.
Displacement mapping: Models that are handcrafted by artists tend to have much higher triangle counts before they are placed into a game, where they must be downgraded to meet the polygon budget. Displacement mapping stores details of the high quality mesh as a texture, that can be applied at run time to restore the fine details, using some LOD scheme as mentioned above.
Subdivision surfaces: Each polygonal mesh has a well-defined smooth surface associated with it. A refinement algorithm computes the smooth surface as the limit of a recursive process (the exact surface depends on the method used). One such method is Catmull-Clark subdivision. A popular approach by Loop and Schaefer [2] is suited for the GPU, and approximates the surface with bicubic patches.
Displaced subdivision surfaces: Lee et al. had the splendid idea of combining subdivision surfaces with displacement mapping [3]. The result is compact storage of fine-detail models, well suited for animation or rendering.
Smoothing 2D surfaces: GUI elements or text can be described by higher order geometry, such as bezier curves, to provide potentially infinite smoothness. Tessellation could be used to generate the geometry to render smooth 2D graphics.
For further reading, we refer to 10 fun things to do with tessellation [4], describing more uses such as terrain or hair rendering. For more information about the basics of tessellation, these articles ([5], [6]) provide a concise introduction.

Displacement mapping

In this sample we apply a displacement map to a spherical mesh, producing cool planetoidal-esque shapes. We begin by procedurally generating a cube mesh, where each face is predivided into a number of quadrilateral patches. A patch is a new primitive type, and defines the points that can be operated on in the tessellation shaders. These patches are further subdivided – into proper triangles – with a level of detail dictated by the control shader. As shown in the figure, we can produce a sphere from the tessellated mesh by normalizing each vertex to unit length.

Figure 2: Each patch of the initial cube is further subdivided into triangles.

Each point in the triangle is then normalized to unit length to produce a smooth sphere. Note that a uniform tessellation on the
cube will be denser near the seams of the faces on the sphere.”

The displacement map is generated using the popular 3D modelling package Blender, by the use of combining procedural textures of different types. To apply the map to the sphere, we need a mapping between vertex coordinates on the sphere and texel coordinates in the texture. Several methods for mapping a sphere exist, each with their own advantages and drawbacks. We chose a cubemap, where each side of the initial cube is mapped to one square in a texture of six.

Figure 3: The displacement map consists of 6 squares, corresponding to each face of the cube.
Note that there are no visible seams between the faces in this figure.

Sampling the cubemap is done by intersecting the sphere normal with the associated cube face. The mathematics for this turn out to be very simple, making cubemaps one of the more efficient mappings. Cubemap texture sampling is available as a hardware-level operation in the shaders.

Care should be taken to avoid visible seams between cubemap faces. Seamless filtering is available as a texture parameter hint in OpenGL, and may avoid issues. Further improvements can be made by averaging edges in the texture beforehand.

Figure 4: The above cubemap applied to the tessellated sphere.

In the evaluation shader, we renormalize the generated vertices and sample the texture using the sphere normal (parallel to the vertex position in our case!). The vertex is finally displaced by an amount proportional to the color of the texel.

Optimizations

While the basics of the displacement mapping technique are apparently simple, a good result does not come along by itself. In the next sections we describe some pitfalls associated with tessellation, as well as some optimizations that can be made to improve performance and visuals.

Backface culling

The tessellation evaluation shader will be run for each vertex generated by the tessellator. However, many of these vertices might end up being invisible when finally rendered to the screen. It is therefore beneficial to determine whether or not we can cull a patch, before submitting it to the tessellator, where further work would be wasted.

We can cull a patch by setting its tessellation factors to zero, effectively generating no additional geometry. This is done in the control shader, by checking whether all the vertices of a patch are either offscreen or are hidden by front geometry. In the case of a perfect sphere, a patch is hidden if all of its normals are facing away from the camera in view-space. That is, the z-component of each normal is negative. However, when the sphere is morphed we may have geometry that is displaced far enough to be visible from behind the sphere. A straightforward fix is to require that the z-component is less than some handpicked bias.

Finally, we project the patch vertices to normalized device coordinates, and compare the vertices with the frustum bounds to determine if the patch is fully offscreen.

Progressive level of detail

Patches that do not cover a large screen area need not be tessellated too much. We take advantage of this to increase performance. The method used in the sample is a naive implementation of screen-space coverage adaptive tessellation, and works as follows:

Project the patch vertices to screen space
Compute the lengths of each projected edge (the unit will be in pixels)
The tessellation level of each edge is computed as a linear function of its screen space length. An edge is maximally tessellated when its length is equal to a handpicked threshold.
The inner tessellation levels are computed as the average of the associated outer levels.
Care must be taken to ensure that the edge levels are computed consistently across all patches. If two neighbouring edges do not have the same tessellation level, horrible gaps or flickering may occur.

Mipmap selection

Tessellation of meshes has close ties with sampling theory. I.e. the generated geometry must have a high enough sampling rate, in order to reconstruct the geometry described by the displacement map. If the sampling rate is too low, we could attempt to reduce the frequency of the displacement map to compensate.

In the sample code, we attempt to do this by selecting a lower quality mipmap of the texture, depending on the tessellation. Mipmaps are pre-calculated, optimized versions of the original texture, each of which downscaled by a factor of two from the previous level. You can think of this as reducing the frequency of the displacement geometry by a half, for each mipmap level.

A clever strategy could be to actually analyze the frequency components of the map, and select an appropriate mipmap level based on the tessellation level. In the sample we simply linearly select between some lower-bounded mipmap and the best mipmap based on camera distance.

Aliasing

It is possible that the displacement map simply has too much detail, than can be represented by the tessellated mesh. The result of this is aliasing, and can be painfully visible.

The following figures demonstrate the phenomenom. In both cases, the displacement map consisted of a single sharp line, dividing the map into a black and white section. In the first figure, the line was aligned with the grid, but slightly offset from the sampling center. In the second figure, the line was not aligned with the grid.

Figure 5: The mesh suffers aliasing in one dimension due to high-frequency components in the displacement map that are insufficiently sampled by the tessellation.

Figure 6: The mesh suffers aliasing in two dimensions due to low sampling rate, and misalignment between the underlying sampling points and the displacement map.

This jarring effect can be somewhat reduced by simply increasing the global tessellation factor, but that approach is not scaleable. Several techniques have been developed for preventing aliasing in tessellation. Such a technique must prevent gaps between patches and mesh swimming (vertices that squirm around when the level of detail is varied by the camera distance). We mention some ideas:

Importance sampling: The generated tessellation points are shifted such that they align better with contours in the displacement map.
Noise and dithering: Hide the effect by adding noise to the displacement map. We employ this strategy in one of our displacement maps that initially had visible artifacts from steep hills.

Figure 7: The effects of aliasing can be somewhat hidden by adding noise to the displacement map.

Geometry adaptive tessellation

If your mesh consists of large flat areas – such as the world’s greatest virtual concrete slab [1] – we can reduce the triangle count with no apparent loss of fidelity. Geometry adaptive tessellation does this by examining the curvature or the underlying surface, and varies the level of tessellation accordingly. A possible approach applied to subdivision surfaces is described in [8].

Final remarks

Hopefully, this sample has demonstrated the potential use cases of GPU accelerated tessellation, as well as what pitfalls that lay before the eager programmer. If the reader decides to go further with tessellation, it is important to consider that the Mali GPU – in its own peculiarity – does not have a dedicated tessellation unit. The performance of tessellation can be highly dependent on the underlying hardware, and should be used with care.

References

[1] The Tech Report. “The world’s greatest virtual concrete slab”

[2] Loop, Schaefer. “Approximating Catmull-Clark Subdivision Surfaces with Bicubic Patches”

[3] Lee, et al.. “Displaced Subdivision Surfaces”

[4] Castaño, Ignacio. “10 Fun Things to do with Tessellation”

[5] The Little Grasshopper. “Triangle Tessellation with OpenGL 4.0”

[6] OpenGL SuperBible. “Primitive Processing in OpenGL”

[7] Rákos, Daniel. “History of hardware tessellation”, available online.

[8] GPU Gems 2. “Adaptive Tessellation of Subdivision Surfaces with Displacement Mapping”

Contents

• Introduction
• Isosurfaces
• Visulizing isosurfaces
• Texturing and shading
• Further reading
• References

This sample uses OpenGL ES 3.1 and the Android extension pack to procedurally generate complex geometry in real-time with geometry shaders.

Introduction

The Android extension pack adds many new features to the mobile platform. Some features have been exclusive to desktop OpenGL for a long time, such as geometry and tessellation shader stages. In this sample, we showcase the use of geometry shaders to generate meshes from procedural surfaces, in real-time.

The source for this sample can be found in the folder of the SDK.

Geometry shaders

The geometry shader is an optional programmable stage in the pipeline, that allows the programmer to create new geometry on the fly, using the output of the vertex shader as input. For example, we could invoke the geometry shader with a set of N points using a single draw call,

glDrawArrays(GL_POINTS, 0, N)

and output a triangle for each such point in the geometry shader,

layout(points) in;
in vec4 vs_position[];
void main()
{
vec4 position = vs_position[0];
gl_Position = position - vec4(0.05, 0.0, 0.0);
EmitVertex();
gl_Position = position + vec4(0.05, 0.0, 0.0);
EmitVertex();
gl_Position = position + vec4(0.0, 0.05, 0.0);
EmitVertex();
EndPrimitive();
}

In this demo, we use geometry shaders to output a variable number of triangles (up to 6), given a set of points in a grid as input. For a more in in-depth introduction to geometry shaders, see [6].

Isosurfaces

In traditional 3D graphics, we are used to thinking about surfaces as a set of connected triangles. Modelling such surfaces involve meticulous placing of points, and connecting them together to make faces. For certain types of geometry, this works alright. But making natural-looking, organic geometry is a very tedious process in this representation.

In this demo, we will take advantage of recent advances in GPU hardware and use a more suitable surface representation, to generate complex surfaces in real-time. Instead of describing our surface as a set of triangles, we will define our surface as the set of points where a function of three spatial coordinates evaluates to zero. In mathematical terminology, this set is known as the isosurface of the function, and the function is referred to as a potential.

It might be difficult to imagine what that would look like, but think of it like this: An elevation map can be considered as a function, which takes a point (x, y) in the plane and gives the height at that point. The set of points where the height is equal to a specific value is called an isoline, because it traces out a curve in the plane. An isosurface is the same concept, but bumped up one dimension.

Isosurfaces are nothing new. They have been commonly used to visualize simulations in computational fluid dynamics, or in medical imaging to visualize regions of a particular density in a three-dimensional CT scan.

Modelling isosurfaces

I’ve criticized triangles as being unfit for modelling organic geometry, so I better prove to you that this isosurface stuff is somehow better! To that end, let’s take a look at how you might make some simple terrain.

Let’s begin with a plane. For that we need a function which is negative below the plane and positive above it. When a point is on the plane, the function is zero.

In code this could be described as

surface = p.y;

where p is the sampling point in 3D space, and surface is the function value as stored in the 3D texture. Terrain usually has some valleys, so let’s add a value that oscillates as the sampling point travels the xz-plane.

surface = p.y;
surface += 0.1 * sin(p.x * 2.0 * pi) * sin(p.z * 2.0 * pi);

Getting there, but this hardly looks like something you would find in nature. A common tool in procedural modelling is to use noise. There are many varieties of noise, such as simplex-, Perlin- or Worley-noise ([8], [9], [10]), that can be used to make interesting perturbations.

surface = p.y;
surface += 0.1 * sin(p.x * 2.0 * pi) * sin(p.z * 2.0 * pi);
surface += 0.075 * snoise(p * 4.0);

This is pretty good. But let’s say we want the terrain to flatten out at the top. To do this, we can intersect the surface with a plane that is facing downwards and cutting the terrain through the tops.

surface = p.y;
surface += 0.1 * sin(p.x * 2.0 * pi) * sin(p.z * 2.0 * pi);
surface += 0.075 * snoise(p * 4.0);
surface = max(surface, -(p.y - 0.05));

We encourage the reader to have a look at Inigo Quilez’s introduction to modelling with distance functions [5]. Our demo doesn’t rely on the potential function being a distance field – that is, the value describes the distance to the closest surface – but the article describes some mathematical tricks that still apply. Nevertheless, this short exercise should have shown how compact we can represent the geometry, and how the representation allows for much easier experimentation, than a conventional triangle description.

Visulizing isosurfaces

Now that we have a sense of what isosurfaces are, and how we can make them, we need a way of drawing them. There are many methods for visualizing isosurfaces; some based on raytracing, some based on generating a polygonal mesh. We do not intend to give an overview of them here.

A famous method in the latter category is the marching cubes algorithm. Its popularity is perhaps best attributed to the splendid exposition by Paul Bourke [4], which does a fine job at explaining the technique, and provides a small and fast implementation as well.

In the demo, we use a different meshing technique known as surface nets. This technique appears to give similar mesh quality as marching cubes, but with considerably less geometry. Less geometry means less bandwidth, which is important to keep your mobile GPU happy.

Naive surface nets

The marching cubes technique was initially published in 1987, and is getting rather old. Not surprisingly, a lot of research has been done on the subject since then, and new techniques have emerged. Surface nets is a technique that was first introduced by Sarah Frisken [1] in 1999. The large time gap between the two techniques might lead you to believe that there may be a similar gap in complexity. But surprisingly enough, the naive version of surface nets [7] is simple enough, that you may even have thought of it yourself! Let’s first take a look at how it works in 2D.

We begin by laying a grid over the surface domain. In every cell of the grid, we sample its four corners to obtain four values of the potential function. If all values have the same sign, then the cell is either completely inside or outside the surface. But if there is a sign change between two corners, then the function must have intersected the edge between them.

If we place a single vertex in the center of each cell that contains an intersected edge, and connect neighbouring cells together, then we get a very rough approximation of the surface. The next step is to somehow smooth out the result. In the original paper, Gibson proposes an iterative relaxation scheme that minimizes a measure of the global surface “roughness”. In each pass of this routine all vertices are perturbed closer to the surface, but kept inside their original box. The latter constraint is important to preserve sharp features on the surface.

However, global optimization like that can be quite hairy to implement in real-time. A simpler approach is to compute the surface intersection point on each edge in a cell, summing them up and computing the average. The figure below shows intersection points as x’s and their average – the center of mass, if you will – as dashes.

The vertex is then perturbed to the center of mass, as shown below. This turns out to work rather well. It gives a smoother mesh, while preserving sharp features by keeping vertices inside their cells.

You can imagine how the technique extends itself to the third dimension. Each cell becomes a cube, and instead of linking neighbouring cells together by lines, they are connected by faces. The figure shows the process for a 3D sphere. On the left we show the mesh generated by linking together surface cells without the smoothing step. On the right, the vertices are perturbed towards the center of mass.

This naive implementation – as it is called [7] – lends itself very well to parallelization, as described in the next section.

Implementing surface nets on the GPU

Our GPU implementation works in three passes:

For each sample point in the grid (i.e. each corner of each cube): Sample the potential function, and store the result in a 3D texture.
For each cube in the grid: Fetch the function value at its 8 corners, and compute the center of mass. Store the result in another 3D texture.
For each cube in the grid that was on the surface: Construct faces by linking together neighbor cells that were on the surface too.
The first two passes are implemented in compute shaders, while the third pass uses a geometry shader to produce faces on-the-fly. It was easy to create links between surface cells for the 2D case above, but it might require a stretch of your imagination to do the same in 3D. So let’s elaborate a bit on that.

Generally, each cube on the surface can connect to its six neighbors with 12 possible triangles. A triangle is only created if both relevant neighbors are also on the surface. But considering all triangles for each cube would give redundant and overlapping triangles. To simplify, we construct faces backwards from a cube, connecting with neighbors behind, below or to the left of it. This means that we consider three edges for each cube. If an edge exhibits a sign change, the vertices associated with the four cubes that contain the edge are joined to form a quad.

To speed up the third pass, we only process cells that we know to be on the surface. This is done by the use of indirect draw calls and atomic counters. In the second pass, if the surface intersects the cell, we write the cell’s index to an index buffer and increment the count atomically.

uint unique = atomicCounterIncrement(outCount);
int index = texel.z * N * N + texel.y * N + texel.x;
outIndices[unique] = uint(index);

Here, N is the sidelength of the grid. When performing the draw call for the geometry shader, we bind a buffer containing the grid of points, the index buffer contaning which points to process, and an indirect draw call buffer contaning the draw call parameters.

glBindBuffer(GL_ARRAY_BUFFER, app->points_buffer); glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, app->index_buffer); glBindBuffer(GL_DRAW_INDIRECT_BUFFER, app->indirect_buffer); glDrawElementsIndirect(GL_POINTS, GL_UNSIGNED_INT, 0);

Texturing and shading

The resulting mesh can look slightly bland. An easy way to add texture is to use an altitude-based color lookup. In the demo, we map certain heights to certain colors using a 2D texture lookup. The height determines the u coordinate, and allow for the use of the v coordinate to add some variation.

Lighting is done by approximating the gradient of the potential function in the geometry shader and normalizing, like so

float v000 = texelFetch(inSurface, texel, 0).r;
float v100 = texelFetch(inSurface, texel + ivec3(1, 0, 0), 0).r;
float v010 = texelFetch(inSurface, texel + ivec3(0, 1, 0), 0).r;
float v001 = texelFetch(inSurface, texel + ivec3(0, 0, 1), 0).r;
n = normalize(vec3(v100 - v000,
v010 - v000,
v001 - v000));

This gives a very faceted look, as the normal is only computed per face. A smoother normal could be computed by approximating the gradient at each generated vertex, and blending between them in the fragment shader. Alternatively – but costly – you could sample the gradient in the fragment shader.

Further reading

In this demo we have taken a look at how we can use potential functions to model complex geometry, and the rendering of such functions using mesh extraction with geometry shaders. There has been much research in the area of mesh extraction, and we have only taken a brief look at one technique.

More advanced meshing techniques, such as dual contouring [3], improve upon the shortcomings of surface nets, and allow for the use of adaptive octrees on much larger grids.

Triplanar projection mapping [2] can be used to map a texture onto all three axes of your geometry, with minimal distortion. It can add more fidelity and variation beyond simple altitude-based color lookups.

References

[1] Sarah F. Frisken Gibson. “Constrained Elastic Surface Nets”

[2] Ryan Geiss. “Generating Complex Procedural Terrains Using the GPU”

[3] T. Ju, F. Losasso, S. Schaefer, and J. Warren. “Dual Contouring of Hermite Data”

[4] Paul Bourke. “Polygonising a scalar field”

[5] Inigo Quilez. “Modeling with distance functions”

[6] open.gl. “Geometry shaders”

[7] Mikola Lysenko. “Smooth Voxel Terrain”

[8] Wikipedia. “Simplex noise”

[9] Wikipedia. “Perlin noise”

[10] Wikipedia. “Worley noise”

ARM® Mali™  GPU Vulkan™ Drivers

These packages contain Vulkan Drivers to use on development platforms featuring the ARM Mali GPU family.

ARM Mali Vulkan Driver Releases:

Vulkan and the Vulkan logo are trademarks of the Khronos Group Inc.

Software Development Kit for ARM Mali Vulkan applications on Android

The Mali Vulkan Software Development Kit is a collection of resources to help you build Vulkan applications for a platform with a Mali GPU and an ARM processor. You can use it for creating new applications, training, and exploration of implementation possibilities.

MALI VULKAN SDK

Mali Vulkan SDK

By downloading the packages below you acknowledge that you accept the END USER LICENCE AGREEMENT FOR MALI VULKAN SDK

---

END USER LICENCE AGREEMENT FOR MALI VULKAN SDK

THIS END USER LICENCE AGREEMENT (“LICENCE”) IS A LEGAL AGREEMENT BETWEEN YOU (EITHER A SINGLE INDIVIDUAL, OR SINGLE LEGAL ENTITY) AND ARM LIMITED (“ARM”) FOR THE USE OF THE DELIVERABLES ACCOMPANYING THIS LICENCE. ARM IS ONLY WILLING TO LICENSE THE DELIVERABLES TO YOU ON CONDITION THAT YOU ACCEPT ALL OF THE TERMS IN THIS LICENCE. BY CLICKING “I AGREE” OR BY INSTALLING OR OTHERWISE USING OR COPYING THE DELIVERABLES YOU INDICATE THAT YOU AGREE TO BE BOUND BY ALL THE TERMS OF THIS LICENCE.

1. DEFINITIONS.

“Authorised Purpose” means; (i) with respect to Deliverables other than the Samples, the right to use internally to execute Vulkan graphics applications and shader programs on ARM based devices; and (ii) with respect to the Samples the right to develop Software Applications.

“Deliverables” means the software binaries and source code accompanying this Licence, and any printed, electronic or online documentation supplied with it, in all cases relating to the Mali Vulkan SDK, expressly excluding the Separate Files.

“Input” means all suggestions, comments, feedback, ideas, or know-how (whether in oral or written form) provided by you to ARM under this Licence and in connection with the Deliverables.

“Separate Files” means the files identified in Clause 4 below, which are subject to the terms of the licence agreements identified in Clause 4.

“Software Applications” means graphics application programs developed by you under this License that conform to the Khronos Vulkan application programming interface.

“Subsidiary” means any company the majority of whose voting shares is now or hereafter owned or controlled, directly or indirectly, by ARM. A company shall be a Subsidiary only for the period during which such control exists.

“Samples” means either or both; (i) any deliverables included in the directory titled “Samples”; and (ii) any files in the directory titled “vulkan-sdk”.

2. LICENCE GRANT.

(i) DELIVERABLES: ARM hereby grants to you, subject to the terms and conditions of this Licence, a nonexclusive, nontransferable, royalty free, worldwide licence to:

(a) use and copy the Deliverables solely for the Authorised Purpose

(b) use, copy and modify any source code provided as part of the Deliverables solely for the Authorised Purpose; and

(c) provided you comply with the terms of Clause 2(ii), distribute and sub-license to third parties the rights to use and copy the Samples, or derivatives thereof created under this Licence, in source or object code form, solely as part of Software Applications.

(ii) FURTHER CONDITIONS ON REDISTRIBUTION AND SUB-LICENSING: If you choose to redistribute or sub-license the Samples or derivatives thereof, made under the licences granted in Clause 2(i) above, you agree; (a) to ensure that they are licensed for use only as part of your Software Applications; (b) not to use ARM’s (or its licensors’) name, logo or trademarks to market your Software Applications; (c) to include valid copyright notices on your Software Applications, and to preserve any copyright notices (whether ARM’s or its licensor’s) which are included with, or in, the Deliverables; and (d) to ensure that your distributors and customers agree that such Deliverables and derivatives shall only be used subject to restrictions which are equivalent to those in Clause 3 except that “you” is replaced by the recipient entity’s name.

2.A Licence to ARM

2.A1 You may at your discretion deliver any Input to ARM.

2.A2 Except as expressly agreed to the contrary in writing by both parties, you hereby grant to ARM and its Subsidiaries under your and your affiliates (as applicable) intellectual property a perpetual, irrevocable, royalty free, non-exclusive, worldwide licence to: (i) use, copy, modify, and create derivative works of the Input as part of any or all of the Deliverables and any other ARM product(s); (ii) sell, supply or otherwise distribute the whole or any part of the Input (and derivative works thereof) as part of any or all of the Deliverables and any other ARM product(s); and (iii) sublicense to third parties the foregoing rights, including the right to sublicense to further third parties. No right is granted by you to ARM to sublicense your and your affiliates (as applicable) intellectual property except to the extent that it is provided to ARM as Input and is embodied in any or all of the Deliverables and any other ARM product(s).

2.A3 Except as expressly licensed to ARM in Clause 2.A2, you retain all right, title and interest in and to the Input provided by you under this Licence.

2.A4 You shall not knowingly give to ARM any Inputyou have reason to believe is subject to any patent, copyright or other intellectual property claim or right of any third party;

2.A5 For the avoidance of doubt, ARM shall be free to use, copy, disclose or otherwise distribute any Input as part of any or all of the Deliverables and any other ARM product(s) to any third party or pursuant to any of the licences granted in Clause 2.A2 without obligation or restriction of any kind.

3. RESTRICTIONS ON USE OF THE DELIVERABLES.

RESULTS: You shall treat any and all bahaviour and performance figures relating to the Deliverables, and any other results of your use and testing of the Deliverables which are indicative if its performance, efficacy, reliability or quality (“Results”) as confidential information, and you shall not disclose such information to any third party without the express written permission of ARM. ARM may agree to waive the restrictions in this paragraph if requested in writing. You agree that the Results are not representative of simulation results that may be obtained on Mali™ hardware.

REVERSE ENGINEERING: Except to the extent that such activity is permitted by applicable law you shall not reverse engineer, decompile or disassemble any of the Deliverables. If the Deliverables were provided to you in Europe you shall not reverse engineer, decompile or disassemble any of the Deliverables for the purposes of error correction.

RESTRICTIONS ON TRANSFER OF LICENSED RIGHTS: The rights granted to you under this agreement may not be assigned, sublicensed or otherwise transferred by you to any third party without the prior written consent of ARM. You shall not rent or lease the Deliverables, or except as expressly provided above, share them with third parties.

TITLE AND RESERVATION OF RIGHTS: You acquire no rights to the Deliverables other than as expressly provided by this Licence. The Deliverables are licensed not sold. ARM does not transfer title to the Deliverables to you. In no event shall the licences granted in Clause 2 be construed as granting you expressly or by implication, estoppel or otherwise, licences to any ARM technology other than the Deliverables.

COPYRIGHT NOTICES: You shall not remove from the Deliverables any copyright notice or other notice and shall ensure that any such notice is reproduced in any copies of the whole or any part of the Deliverables made by you.

4. THIRD PARTY RIGHTS

The Deliverables contain files licensed from third parties which are subject to the relevant terms and conditions set out in the Schedule to this Licence (“Third Party Terms”). The Separate Files are delivered to you as part of the Deliverables for your convenience, and are expressly subject to the Third Party Terms. You hereby agree to comply with all the terms and conditions imposed on you by the Third Party Terms. Further you hereby agree that ARM shall have no liability to you under the terms of this Licence in respect of the Separate Files.

5. NO SUPPORT.

ARM has no obligation to support or to continue providing or updating any of the Deliverables.

6. NO WARRANTIES.

YOU AGREE THAT THE DELIVERABLES ARE LICENSED “AS IS”, AND THAT ARM EXPRESSLY DISCLAIMS ALL REPRESENTATIONS, WARRANTIES, CONDITIONS OR OTHER TERMS, EXPRESS OR IMPLIED, INCLUDING WITHOUT LIMITATION THE IMPLIED WARRANTIES OF NON-INFRINGEMENT, TITLE, SATISFACTORY QUALITY, MERCHANTIBILITY, AND FITNESS FOR A PARTICULAR PURPOSE.

YOU EXPRESSLY ASSUME ALL LIABILITIES AND RISKS, FOR USE OR OPERATION OF ANY APPLICATION PROGRAMS YOU CREATE WITH THE DELIVERABLES, AND YOU ASSUME THE ENTIRE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.

7. LIMITATION OF LIABILITY.

TO THE MAXIMUM EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL ARM OR ITS LICENSORS BE LIABLE FOR ANY INDIRECT, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES (INCLUDING LOSS OF PROFITS) ARISING OUT OF THE USE OR INABILITY TO USE THE DELIVERABLES WHETHER BASED ON A CLAIM UNDER CONTRACT, TORT OR OTHER LEGAL THEORY, EVEN IF ARM OR ANY SUCH LICENSOR WAS ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

ARM does not seek to limit or exclude liability for death or personal injury arising from ARM’s negligence and because some jurisdictions do not permit the exclusion or limitation of liability for consequential or incidental damages the above limitation relating to liability for consequential damages may not apply to you.

NOTWITHSTANDING ANYTHING TO THE CONTRARY CONTAINED IN THIS LICENCE, THE MAXIMUM LIABILITY OF ARM TO YOU IN AGGREGATE FOR ALL CLAIMS MADE AGAINST ARM IN CONTRACT TORT OR OTHERWISE UNDER OR IN CONNECTION WITH THE SUBJECT MATTER OF THIS LICENCE SHALL NOT EXCEED THE GREATER OF THE TOTAL OF SUMS PAID BY YOU TO ARM (IF ANY) FOR THIS LICENCE AND US$5.00.

8. U.S. GOVERNMENT END USERS.

US Government Restrictions: Use, duplication, reproduction, release, modification, disclosure or transfer of this commercial product and accompanying documentation is restricted in accordance with the terms of this Licence.

9. TERM AND TERMINATION.

This Licence shall remain in force until terminated by you or by ARM. Without prejudice to any of its other rights if you are in breach of any of the terms and conditions of this Licence then ARM may terminate this Licence immediately upon giving written notice to you. You may terminate this Licence at any time.

Upon termination of this Licence by you or by ARM you shall stop using the Deliverables and destroy all copies of the Deliverables in your possession together with all documentation and related materials.

The provisions of Clauses 1, 2.A2 – 2.A5, 4, 7, 8, 9, and 10 shall survive termination of this Licence.

10. GENERAL.

This Licence is governed by English Law. Except where ARM agrees otherwise in a written contract signed by you and ARM, this is the only agreement between you and ARM relating to the Deliverables and it may only be modified by written agreement between you and ARM. Except as expressly agreed in writing, this Licence may not be modified by purchase orders, advertising or other representation by any person. If any clause in this Licence is held by a court of law to be illegal or unenforceable the remaining provisions of this Licence shall not be affected thereby. The failure by ARM to enforce any of the provisions of this Licence, unless waived in writing, shall not constitute a waiver of ARM’s rights to enforce such provision or any other provision of this Licence in the future.

You agree to comply fully with all laws and regulations of the United States and other countries (“Export Laws”) to assure that the Deliverables are not (1) exported, directly or indirectly, in violation of Export Laws, either to any countries that are subject to U.S.A. export restrictions or to any end user who has been prohibited from participating in the U.S.A. export transactions by any federal agency of the U.S.A. government; or (2) intended to be used for any purpose prohibited by Export Laws, including, without limitation, nuclear, chemical, or biological weapons proliferation.

Schedule – Third Party Terms

KHRONOS
Copyright (c) 2015 The Khronos Group Inc.

Permission is hereby granted, free of charge, to any person obtaining a
copy of this software and/or associated documentation files (the
“Materials”), to deal in the Materials without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Materials, and to
permit persons to whom the Materials are furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Materials.

THE MATERIALS ARE PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
MATERIALS OR THE USE OR OTHER DEALINGS IN THE MATERIALS.

This header is generated from the Khronos Vulkan XML API Registry.
OPENGL MATHEMATICS (GLM)

Copyright (c) 2005 – 2015 G-Truc Creation

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

CMake – Cross Platform Makefile Generator

Copyright 2000-2009 Kitware, Inc., Insight Software Consortium
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the names of Kitware, Inc., the Insight Software Consortium, nor the names of their contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
STB_IMAGE
https://github.com/nothings/stb/blob/master/stb_image.h

These libraries are in the public domain (or the equivalent where that is not possible). You can do anything you want with them. You have no legal obligation to do anything else, although I appreciate attribution.

This software is in the public domain. Where that dedication is not recognized, you are granted a perpetual, irrevocable license to copy, distribute and modify this file as you see fit.

ARM contract reference: LES-PRE-20857

Linux Archive [TGZ]

SIGN UP FOR UPDATES (modal)

Sign up for the latest updates on the Mali Developer Tools, SDKs, Tutorials and Developer Guides.

• First Name*
• Last Name*
• Email*
• Company*
• Terms and Conditions
• No thanks (close)
• Comments
  This field is for validation purposes and should be left unchanged.

DOWNLOAD v1.0

MALI VULKAN SDK TUTORIALS

Support

HTML Documentation included with installer package

Please submit your questions and issues to the Mali Developer Forums

The Android and Linux version of the Mali GPUs Device Driver provide low-level access to the Mali-G71 GPU.

Some of these components are being made available under the GPLv2 licence. This page provides access to the source packages from which loadable kernel modules can be built.

Note that these components are not a complete driver stack. To build a functional OpenGL ES you need access to the full source code of the Mali GPU DDK, which is provided under the standard ARM commercial licence to all Mali GPU customers.

The open source drivers provided on this page are designed to run with a version-compatible release of the Mali GPU DDK. In functional and performance terms they are identical to the implementations provided under the commercial licence. By also releasing them under the GPLv2 licence we hope to make it easier to include Mali GPU drivers on any Linux or Android platform.

CURRENT RELEASE INFORMATION

By downloading the packages below you acknowledge that you accept the End User License Agreement for the Mali GPUs Kernel Device Drivers Source Code.

GPU Kernel Device Drivers

Android Gralloc Module Source Code For Mali-200/300/400/450, Mali-T6xx/T7xx/T860/T880 and Mali- G71 GPUs. Linux/Android Gralloc Module Under Apache License.

This is an Android Gralloc module which allocates memory from the Unified Memory Profider (UMP) device driver. The Gralloc module is used by Android’s composition engine in order to allocate and manage memory which is suitable for 2D and 3D graphics usage. The Mali GPU driver stack can use UMP memory directly, resulting in zero-copy operations when compositing.

Note that this component is not a complete driver stack. To build a functional OpenGL ES or OpenVG driver you need access to the full source code of the Mali GPU DDK, which is provided under the standard ARM commercial licence to all Mali GPU customers. For a complete integration of the Mali GPU DDK with the Android environment refer to the Integration Guide supplied with the Mali GPU DDK.

The open source code provided on this page is designed to run with a version-compatible release of the Mali GPU DDK. By releasing this software under the Apache licence we hope to make it easier to include Mali GPU drivers in any Android platform.

FEATURES

The Android Gralloc module handles:

• Access to allocated UMP memory through a secure ID. This enables memory to be shared across different applications, drivers and hardware components to facilitate zero-copy operations
• The physical address information required to set up an MMU or MPU table
• A method to map UMP memory into CPU address space, to enable reading and writing

Mali GPUs Android Gralloc Module Source Code

By downloading the packages below you acknowledge that you accept the Apache License Agreement for the Mali GPUs Android Gralloc Module Source Code

This Vulkan Sample Code introduces depth testing and push constants.

By building on the Rotating Texture sample, the sample illustrates the use of push constants and how to setup various Vulkan objects to enable depth testing.

The source for this sample can be found in samples/spinning_cube in the Mali Vulkan SDK.

Vulkan Sample Code: Spinning Cube

This Vulkan Sample Code introduces efficient Multisampling.

A look at how to implement multisampled anti-aliasing (MSAA) on Mali GPUs in the most efficient way. There are two main approaches to choose from, where one alternative is dramatically better than the other.

The sample is based on the Rotating Texture sample to focus on the differences from rendering without MSAA to rendering with MSAA.

The source for this sample can be found in samples/multisampling in the Mali Vulkan SDK.

Vulkan Sample Code: Multisampling

This Vulkan Sample Code gives an introduction to Compute Shaders.

In Vulkan, Compute Shaders have first class support and give applications the ability to perform non-graphics related tasks on the GPU. The sample will demonstrate a very simple particle system moving particles around on the GPU and rendering them to screen as additive blended point sprites.

Note: This sample assumes you have some knowledge of how compute shaders work in other related graphics APIs such as OpenGL ES.

The source for this sample can be found in samples/basic_compute in the Mali Vulkan SDK.

Vulkan Sample Code: Compute Shaders

This Vulkan Sample Code uses Vulkan to draw a scene using multiple CPU threads.

Spreading the rendering workload across multiple CPU threads can have huge benefits for complex applications. In this sample, Rotating Texture is expanded by rendering several textured quads, split over multiple threads and many draw calls.

The source for this sample can be found in samples/multithreading in the Mali Vulkan SDK.

Vulkan Sample Code: Multithreading

This Vulkan Sample Code introduces texturing and uniform buffers to build on the Hello Triangle sample. It also adds texturing, as well as rotating the quad using a uniform buffer.

The source for this sample can be found in samples/rotating_texture in the Mali Vulkan SDK.

Vulkan Sample Code: Rotating Texture

This Vulkan Sample Code introduces ‘Hello Triangle’, a basic Vulkan application that draws a simple triangle to the screen. There are a lot of concepts to internalize.

The source for this sample can be found in samples/hellotriangle in the Mali Vulkan SDK.

Vulkan Sample Code: Hello Triangle

As high performance and high efficiency are increasingly demanded on mobile devices, developers are looking to the Vulkan API for the future of mobile graphics and compute. ARM’s Vulkan conformant Mali GPUs are compatible with the following Vulkan Tools:

• Detect Vulkan performance issues frame-by-frame.
• Redesigned Assets View: show only the data you need.
• Full trace of the Vulkan API: See the order and types of Vulkan calls.
• An invaluable tool for Vulkan development on ARM Mali platforms.

• Resources to help developers build Vulkan applications.
• Full documentation, framework, sample codes and tutorials.
• Create new apps and explore Vulkan development.
• Build for platforms that run Vulkan on Android with a Mali GPU.

Developing a Mobile Virtual Reality application requires a different approach to ‘traditional’ app development. Users are immersed while using a VR application and expect objects and the world around them to behave much like the real world. Mobile VR applications should be built with great care and attention to detail, and must be able to achieve the highest performance possible so that users have a comfortable and memorable experience. This is where our VR Tools can help.

How can I start developing for mobile VR?
The Mali VR SDK contains all you need to start developing VR applications for platforms using an ARM Mali GPU, from tutorials to sample codes.

How can I ensure the best performance of my mobile VR application?
Mali Graphics Debugger allows you to optimize performance on ARM Mali GPUs by checking for any issues or bottlenecks frame-by-frame.

• Detect VR performance issues frame-by-frame.
• Frame capture continues until all render passes started have been completed.
• Colour-coded EGL contexts in Trace, Trace outline and Timeline views.
• An invaluable tool for VR development on ARM Mali platforms.

• Resources to help developers build VR applications.
• Full documentation, framework, sample codes and tutorials.
• Create new apps and explore VR development.
• Build for Android platforms with a Mali GPU.

The downloadable package include documentation and binary builds of the library for multiple platforms:

• Linux 32-bit (ARMv7a)
• Android 32-bit (ARMv7a)
• Android 64-bit (ARMv8a)

For any questions email: developer@arm.com

In order to be able to download the ARM Computer Vision and Deep Learning performance library you need to read and accept the terms of the license below.

• Support for texture parameters, texture layers and sampler parameters.
• Redesigned Assets View: show only the data you need.
• Understand which vertex and fragment shaders are the most expensive.
• An invaluable tool for Unreal Engine development on ARM Mali platforms.

• Resources to help developers build applications.
• Full documentation, framework, sample codes and tutorials.
• Built on a simple framework that can be easily integrated.
• Samples include Multiview rendering, Tessellation and more.

Introduction

If you are developing an android application or game sometimes you can run into issues. When this happens they usually fall into one of two categories: The graphics in my application don’t look like I imagined them, or the speed or frame rate of the application is not good enough for the playing experience to be enjoyable by the end user. There are tools to help you through these issues and this guide is going to show you how and when to use which tools to get the results that you want.

Download the correct tools for the job

If your issue is performance related then you should start by downloading ARM Streamline.  This can be found at https://developer.arm.com/products/software-development-tools/ds-5-development-studio/downloads. This tool allows you to quickly identify where your performance bottleneck is. From there, if you find you’re CPU bound then you should stay with Streamline as it can give you a wealth of information about where CPU time is spent on your application.

However, if you find you are graphics bound or have a graphics related defect, then you can move to the Mali Graphics Debugger (MGD) as it’s the best tool for debugging and optimizing graphics applications.

Setup

So, we are at the point where we know what the problem is and we have selected MGD to help us solve it. First you need to download it from: http://malideveloper.arm.com/resources/tools/mali-graphics-debugger/

The tool works on Windows, Linux and Mac so just pick the version that works for you. From there you can choose to use MGD on a rooted or non-rooted device.

MGD on a rooted device

The simple steps to set this up can be found in the user guide supplied with MGD. The idea is that you will replace your graphics driver on your device with the MGD library. The MGD library will then call the graphics driver on behalf of the application, profiling all the applications on a device without needing to modify any of the source code.

MGD on a non-rooted device

If your device is not rooted then you need to include the MGD in your application. We have various guides available depending on which game engine you are working with:

• If you are creating the application with Unity then follow this guide: https://www.community.arm.com/graphics/b/ece70eab2a564598933fdbde95b81196/posts/building-a-unity-application-with-mali-graphics-debugger-support
• If you are creating the application with Unreal Engine then follow this guide: https://www.community.arm.com/graphics/b/ece70eab2a564598933fdbde95b81196/posts/building-an-unreal-engine-application-with-mali-graphics-debugger-support
• Finally, if you are creating your application with a custom engine then you should follow this guide: https://www.community.arm.com/graphics/b/ece70eab2a564598933fdbde95b81196/posts/using-mali-graphics-debugger-on-a-non-rooted-device

Let’s generate a trace

Okay so now we have our application on the target and we are ready to start to generate a trace. Start up the MGD application on your device and start the deamon with the little toggle on the top.

Now connect from the host application where a blank trace should have been created. MGD is now ready for you to start tracing. Click on the application on the MGD app, your application should now spring into life and MGD should start populating the overview panel and the trace panel with information.

Let the application play out until you get to the area in your application that you are interested in and then pause it using MGD. Once this is complete we are going to do 4 things:

• A frame capture: To do this you need to just click the capture icon. MGD will then run the next frame and capture all of the frame data that is drawn per draw call. This is great to see how your scene is composed.
• An overdraw capture: To do this you need to click on the overdraw icon which should be two to the right of the frame capture icon. Then when this button is selected press the frame capture button again. MGD will then do an Overdraw capture. This is similar to a frame capture in that MGD captures all of the frame data that is drawn per draw call, but this feature also replaces your shader with an almost transparent white. This means the whiter the image the more times you have drawn to the same pixel. If there are areas in your scene that are over 3X overdraw then this is a great place to start optimizing.

• Shader map capture: To do this you need to click on the shadermap icon which should be to the right of the overdraw icon. When this button is selected you then press the frame capture button again. MGD will then do another capture but this time will replace each of your fragment shaders with a unique colour. This will allow you to see which shaders are responsible for drawing different areas of the screen.
• Fragment count: To do this you need to click on the fragment count icon which should be to the right of the shader map icon. When this button is selected you then press the frame capture button again. This will allow MGD to capture all the data from the next frame and while it is doing that it will count how many fragments are drawn to the screen. As MGD already knows how expensive a shader is it means MGD can estimate how many cycles each shader spent on the GPU.

We recommend that you do these 4 things on all traces that you run with MGD whether you are looking for graphics imperfections or you just want to increase performance.

What is wrong (Graphics defects)

So you have graphics imperfections, how can the Mali Graphics Debugger help you? It depends where your imperfection is but here are a few suggestions that you can try:

• If a particular object is not displaying correctly in your scene you could use the Frame Capture above to find out which draw call is the one that is creating the defect. This quickly narrows down your problem area so you can immediately start trying to fix it.
• Once you know which draw call is producing the issue you can use the Geometry Viewer to make sure the geometry that is being sent to the draw call is correct. If this is wrong then you know that this is a problem with the vertices or indices that you are passing to the draw call. If all of this data is correct the problem must be elsewhere.
• If you have a frame that is working correctly and one that then shows you an issue you can select both of the Frames and then right click and select Generate Diff. This will tell you the differences in OpenGL ES state between the two points. If these are different then this is definitely something to look into.
• MGD always does error checking for all api calls so it is easy to pinpoint when there is an error. This also extends to the shaders in your application if they don’t compile or link correctly.

What is wrong (Performance)

If you are looking to improve the performance of your application then you first need to work out whether you are fragment bound, vertex bound, bandwidth bound or CPU bound.  As mentioned earlier, the best tool to work out which one you are is Streamline and you should start your analysis there. The rest of this guide assumes you know which one you are and how you can use MGD to resolve the issue.

Insert a photo here

Vertex bound

Being Vertex bound is uncommon but does happen when you are either drawing too many vertices or you have too many instructions or cycles in your vertex shader. The Trace Outline view gives you an indication of how many vertices are in each of your frames so the best way to start analysing your problem is to select which frame has the most. Then, if you select the shaders view, MGD will tell you which shaders contribute the most to the scene. It is these you want to start to optimize as they will make a difference in your application’s overall performance.

Fragment bound

If your application is fragment bound then there are a few main causes. The first is that you are drawing too many fragments. To reduce this you simply need to reduce the resolution of your application. If you are rendering to a 2560 x 1440 screen that doesn’t mean you need to use that resolution on your game. You could easily switch to 1920 x 1080 and be drawing to just slightly more than half as many fragments.

The next cause could be that you are drawing to the same pixel more than once on the screen. This is known as overdraw. Use the overdraw capture mode described earlier in this article. The area that has the highest overdraw factor should be the easiest to reduce. If you reduce your average overdraw factor from 6x to 3x then, again, you have reduced your fragment load by half.

The final cause is much the same as the vertex one and is that your fragment shaders are simply too expensive. If you take the fragment count capture that you did earlier you can look at the shaders view and see which shaders contribute the most to the scene. It is these you want to start to optimize as they will make the biggest difference to your application’s performance.

Bandwidth bound

If you are bandwidth bound then you should start to look at reducing the size of your textures by including texture compression in your application. MGD will list all your textures in your application as well as whether they are currently being compressed and their current resolution. If you need help in compressing your textures you should use the Texture Compression Tool which can be found at: http://malideveloper.arm.com/resources/tools/mali-gpu-texture-compression-tool/.

You should also start to use Vertex Buffer Objects, again MGD will show you all of the information about the buffers you are currently using.

For more information and resources please see the links below including some of our ARM Tools blogs.

• Draw-call by Draw-call stepping
  To identify draw call related issues, redundant draw calls and other opportunities to optimize
• Texture View
  Visualize an application’s texture usage, to identify opportunities to compress textures or change formats.
• Shader Statistics
  Understand which vertex and fragment shaders are the most expensive with cycle count reporting
• and more…

• • Samples & Tutorials
    Numerous samples from the basic set-up to more complex examples demonstrating shader effects.
    Samples demonstrating features of OpenGL ES 2.0 and OpenGL ES 3.x.
  • Visual Studio Projects
    Project files for all samples supplied for Visual Studio.
    Make files also provided for command line building
  • Simple Framework
    A light weight framework with common math functions, text displaying and timing to see your results quickly.

Developers are finding new ways of creating immersive experiences with mobile VR devices, and Mali GPUs are at the forefront of mobile VR development. Mali T7xx and T8xx GPUs power many mobile VR devices – from headsets that require a mobile phone, to all-in-one devices with built-in screens, operating systems and controls. As mobile VR moves forward, devices featuring Mali-G71 will be able to push boundaries in graphical performance and power efficiency.

Mobile VR currently falls into two categories, All-in-one and Smartphone-based. Below are some examples of each:

All-in-one

Screen, GPU and processor built in.

Deepoon M2

• Mali-T760 (MP8)
• Samsung Exynos 7420
• 2560 x 1440 pixel resolution
• 110 degree Field Of View

Nibiru Pro One Plus 

• Mali-T880 (MP12)
• Samsung Exynos 8890

Smartphone-based

Screen, GPU and processor from mobile.

Samsung Gear VR 

• Mali-T760 (MP8) / T880 (MP12) GPU
• 2560 x 1440 pixel resolution
• 110 degree Field Of View

Google Daydream View 

• Simple, high-quality virtual reality.
• With intuitive controller
• Compatible with Google Daydream and Cardboard apps