<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by nicole on Medium]]></title>
        <description><![CDATA[Stories by nicole on Medium]]></description>
        <link>https://medium.com/@ConnectCode?source=rss-8d47150825f7------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*Kq2YcyRx6gXPTg6i.png</url>
            <title>Stories by nicole on Medium</title>
            <link>https://medium.com/@ConnectCode?source=rss-8d47150825f7------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 20:19:41 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@ConnectCode/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[RealityKit Terrain Shader]]></title>
            <link>https://medium.com/antaeus-ar/realitykit-terrain-shader-4cb508193903?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/4cb508193903</guid>
            <category><![CDATA[terrain-generation]]></category>
            <category><![CDATA[shaders]]></category>
            <category><![CDATA[ios-development]]></category>
            <category><![CDATA[augmented-reality]]></category>
            <category><![CDATA[realitykit]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Mon, 12 Jun 2023 13:08:54 GMT</pubDate>
            <atom:updated>2023-06-15T23:55:14.362Z</atom:updated>
            <content:encoded><![CDATA[<h4>This project shows you how to use Fractal Brownian Motion (fBM) to generate an Augmented Reality (AR) Terrain using iOS RealityKit Custom Materials. RealityKit Custom Material includes a surface shader and geometry modifier, also known as fragment and vertex shaders in OpenGL.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*P9tICjZJnYF4T6z4.gif" /><figcaption>Terrain Shader wih RealityKit Surface Shader and Geometry Modifier</figcaption></figure><p><a href="https://github.com/Misfits-Rebels-Outcasts/RealityKit-Terrain-Shader"><strong>Github Project</strong></a></p><h3>Fractal Brownian Motion (fBM)</h3><p>fBM (Fractal Brownian Motion) is used extensively in computer graphics to model natural phenomenons and textures such as terrain , coastline, fire, smoke, clouds, water flow, water waves , rapids, dinosaur skin, wrinkles etc. It is quite amazing to note how such a simple random function can be used in so many areas to emulate the various forms of nature.</p><p>This project shows how you can use the fBM function to very simply generate a terrain using iOS RealityKit custom materials (which includes the surface shader and geometry modifier, also known as fragment and vertex shaders in OpenGL)</p><p><strong>Parameters</strong></p><p>The form of fBM function we are using has mainly the parameters</p><ul><li>uv — texture coordinates (a value of [0,0] to [1,1], representing the four corners of an image we will like to generate)</li><li>time — in seconds (this value will animate the fBM generate image with respect to time)</li></ul><p><strong>Properties of the fBM</strong></p><p>A fBM can return a single value (one dimensional value) or multiple values (e.g a 3 dimensional vector). You can view a fBM as a random number generator that depends on the current pixel position of the image. However, it is not completely random because the function varies quite smoothly with respect to its neighbours.</p><p>So if the user pass in uv-coordinate values of (0.10,0.10) and (0.11,0.10) the returned value should be quite near to each other. For a 3 dimensional vector , the value of each component will be quite similar to its neighbour’s.</p><p>Apart from varying smoothly with respect to space (i.e position or uv-coordinates), the fBM also varies quite smoothly with respect to time. So a returned value at time = 0.10 sec will be quite similar to a value at time = 0.11 sec. This gives it the property of having a smooth animation w.r.t. time.</p><p>Furthermore, the continuity of many implementations fBM is not restricted to just continuity of values, but also the continuity of its gradient as well as the continuity of (radius of) curvature. This make the fBM function very ‘smooth’ and thus very suitable for artistic renderings.</p><p>A fBM has fractal like properties. If you zoom in to small area, you will be able to notice similar patterns when the image has not been zoomed in.</p><h3>Generating the Terrain</h3><p>This article is mainly about using fBM to generate a terrain with RealityKit 2. The language is in Swift and Metal.</p><p>1.The steps involved is to first generate a 2D plane with 64x64 cells.</p><pre>let plane = buildMesh(numCells: [64, 64], cellSize: 0.01)</pre><p>2. Assign the plane with a Custom Material. This custom material will automatically associate a Surface Shader function and a Geometry Modifier function to the plane</p><pre>    let geometryModifier = CustomMaterial.GeometryModifier(named: &quot;geometryModifier&quot;,<br>                                                            in: library)<br>                          <br>    let surfaceShader = CustomMaterial.SurfaceShader(named: &quot;surfaceShader&quot;,<br>                                                        in: library)<br><br>    let customMaterials = try! CustomMaterial(from: baseMaterial,<br>                                                surfaceShader : surfaceShader,<br>                                                geometryModifier: geometryModifier)<br><br>    let plane = buildMesh(numCells: [64, 64], cellSize: 0.01)<br>                                      <br>    landEntity = ModelEntity(mesh: plane, materials : [customMaterials])</pre><p>3. The geometryModifier and surfaceShader is currently empty functions written in the Metal language. Running the app shows an empty plane.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*prRAaYHoXOT3hcrPWShGZQ.png" /></figure><p>4. Add normals to the plane using the computeNormal function. At this time, we will like to introduce the fBMheightmapx function. This is the main fBM function used for generating a height map for the terrain. In normal situations, we will simply render this height map and we will get a terrain. However, for RealityKit surface shader, it is easier to add details (roughness) to the terrain using normals. This is achieved by computing normals of the height map with the computeNormal function in the surfaceShader function.</p><pre>    //fBMheightmapx  - generates a height map at each position (uv position) of the shader.<br>    //This height-field has a normal at each uv coordinate position pointing away perpendicularly from the surface.<br>    //The normals is computed by offsetting the height-map a little in the x-direction and a little in the y-direction to compute the difference.<br>    //The normals computed thus does not necessary has magnitude / length of 1 (and need to be normalised to length 1).<br><br>    float3 normal = computeNormal(uvscale,time*timefactor/divfactor ,warpType,animVelocity,angle,screensize);<br>                            <br>    normal = normalize(normal);<br>                          <br>    params.surface().set_normal(normal); //set tangent space normal</pre><p>With the normals set, we now run the app and we get something that look like below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*zwJrtqaXN-fQkMgr.png" /></figure><p>5. We now elevate the plane vertically at the various vertices of each cell randomly to produce a an uneven surface. The random values (elevation levels) generated for this terrain is produced directly using the fBM function we mentioned earlier (fBMheightmapx). To elevate the plane, we have to use a geometry modifier (also known as vertex shader) to modify the position of each of these vertices. To do so, we define a geometryModifier function with the following code.</p><pre>   float4 val =<br>                fBMheightmapx(uvscale,time ,warpType,animVelocity,angle);<br><br>    //here , we are preparing the values for a displacement map<br>    //so that only the y-value (height) is modified, but not the x and z value<br>    //the value 0.063 is chosen by trial and error<br>    float3 offset = float3(0.0,0.063*val.r,0.0);<br>                         <br>    //this function applies the fBM generated offset to the plane to make it uneven<br>    params.geometry().set_model_position_offset(offset);</pre><p>We now get something that looks like below. The terrain is almost complete.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*sergMQYmIdyQbSXv56lMrQ.png" /></figure><p>6. For the final step, wee proceed to color the terrain. The fBM function we mentioned earlier can do many things, apart from producing a height map (and its associated normals), we can also use it to color the terrain by mapping each height value with a color.</p><p>For our case, we use a modified extended version of the fBM function named fBMFlow. This function applies a color palette to the fBM generated height-map and retrieve the color at the current uv position. This color is then used to colorize the terrain.</p><pre>    float4 val = fBMFlow(uv, float2(2048,2048), float3(0,0,0), time*timefactor);<br>                                     <br>    half3 color1 = half3(halfr,halfg,halfb);<br><br>    params.surface().set_base_color(color1);</pre><p>With this done, we will get the terrain as shown below. (The image has been enhanced a little for better visibility).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RX_5RuHuSrJN_8IT.png" /></figure><h3>Open Pipeline</h3><p>RealityKit Terrain Shader is part of a community project using the <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef">Open Pipeline</a> to build the next generation of graphics design software. Open Pipeline reduces Cognitive Overload in graphics design with a simplified digital compositing node graph. It is to the user interface of graphics design software like Markdown is to HTML.</p><p><em>Originally published at </em><a href="https://photorealityar.com/realitykit_terrainshader_surfaceshader_geometrymodifier.html"><em>https://photorealityar.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=4cb508193903" width="1" height="1" alt=""><hr><p><a href="https://medium.com/antaeus-ar/realitykit-terrain-shader-4cb508193903">RealityKit Terrain Shader</a> was originally published in <a href="https://medium.com/antaeus-ar">Antaeus AR</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Photo Curves: 100 days of photo editing via Digital Compositing on iPhone]]></title>
            <link>https://medium.com/@ConnectCode/photo-curves-100-days-of-photo-editing-via-digital-compositing-2a1cc09c62e0?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a1cc09c62e0</guid>
            <category><![CDATA[photo-filters]]></category>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[iphone]]></category>
            <category><![CDATA[photo-editing]]></category>
            <category><![CDATA[image-post-production]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Mon, 08 May 2023 11:03:12 GMT</pubDate>
            <atom:updated>2023-05-11T01:01:26.281Z</atom:updated>
            <content:encoded><![CDATA[<h3>A photo taken from a digital camera may sometimes appear too dull or too bright. This is a situation where we could use a photo curve for photo post-processing to make our photos look dramatically better. Applying a curve to a photo can be thought of as a way of remapping an image’s tonality, enabling us to adjust the brightness, increase contrast, or emphasize colors.</h3><h3>Background</h3><p>This article is part of a 100 days series that explores the use of <a href="https://photorealityar.com/digitalcompositing.html"><strong>Digital Compositing</strong></a> for Photo Editing and Photo Post-Processing.</p><h3>Why bother?</h3><blockquote>“The ability to express what we want to do Succinctly”</blockquote><p>Using Digital Compositing has a significant impact on many things including: the way we design graphics software, how we write articles to discuss image post-processing, and how it can help people quickly understand a graphical process. Find out more at the following:</p><p><a href="https://photorealityar.com/digitalcompositing.html">100 Days of Digital Compositing for Photo Editing on iPhone</a></p><h3>Photo Curve — RGB Tone Curve</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*0ui9zxUQRPx5YXUi.png" /></figure><p>Every photo has a mix of shadows, mid tones, and highlights. Shadows are the darkest parts of the photo, highlights are the brightest, and mid tones are in between. When shown on a Photo Curve, the upper-right area of the graph represents the highlights, the lower-left area represents the shadows, and the middle part represents the mid tones.</p><p>The Photo Curve below shows a linear curve mapping all parts of the photo as it is, performing no adjustments to the photo. This means if we apply a Linear Photo Curve to an image, we will be getting back the original image without any changes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*oEf2w3Vtmi6828oG.jpeg" /><figcaption>Linear Photo Curve</figcaption></figure><p>A photo curve can be applied to all the RGB channels together in an image, or to each of the Red, Green, or Blue channel individually. We start by applying the curve on all the channels of a RGB photo.</p><h3>Increase the darkness of a photo</h3><p>As described earlier, the lower left of the Photo Curve of an RGB image represents the shadows, or darkness portion of our photo. By shifting the points on the lower left of a linear curve to the right, we can increase the darkness of the photo. This is shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ff0q6fKgA86CUp4X.jpg" /><figcaption>Tone Curve to increase the darkness</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*bNEaIwCyS6CmzC_x.png" /><figcaption>Original</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2am0DhLBBIJAqhsS.jpg" /><figcaption>After</figcaption></figure><h3>Lighten the shadows or darkness of a photo</h3><p>The shadows or darkness of the photo can be lighten if we shift the lower left points of the Photo Curve up.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*D0Jw1nIa_fdzqpxJ.jpeg" /><figcaption>Tone Curve to lighten the darkness</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TpIA_qw8CDsuvG8Y.png" /><figcaption>Original</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*P1pypafMYceyNcu7.jpg" /><figcaption>After</figcaption></figure><h3>Darken the highlights or brightness of a photo</h3><p>On the other hand, the upper right of a curve represents the highlights or brightness of a photo. We can darken the highlights by shifting the points on the top right down as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5dP_kBTGm4d8y2s4.jpeg" /><figcaption>Tone Curve to darken the highlights</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Mvf-yY6CNvEf0EnA.png" /><figcaption>Original</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*HRKVn9iZq13jnJ05BwB0Lw.jpeg" /><figcaption>After</figcaption></figure><h3>Decrease the contrast of a photo</h3><p>We can decrease the contrast of a photo by a curve shown below. We shifted point 1 up slightly while point 4 down slightly. This has the effect of increasing the darkness of shadows and reducing the brightness of highlights.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*5D3P0oMTp6fsz7xt.jpeg" /><figcaption>Decrease Contrast Curve</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*RTOH8JyfxsKYBkWj.png" /><figcaption>Original</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AkotfAWLGSCSu5Nx.jpg" /><figcaption>After</figcaption></figure><h3>Posterize a Photo</h3><p>We can posterize a photo by reducing the number of colors with a curve shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*M88Ef4jCMvE41hSb.jpeg" /><figcaption>Posterize Curve</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*HJp0LL9UGelXZ2-d.png" /><figcaption>Original</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*aABzLxARVJxsMzzl.jpg" /><figcaption>After</figcaption></figure><h3>RGB Channels</h3><p>Besides applying a Photo Curve on all 3 color channels (RGB). We can also apply the curve on an individual channel such as the Red Channel. We will need to change the Channel Mode from “RGB — All” to “RGB”. This enables us to select the channel to apply our curve to.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*11MmO5dor7f4fuIg.jpeg" /><figcaption>Channel Mode</figcaption></figure><p>In “RGB” Channel Mode, after applying the curve to the individual channels: Red, Green, and Blue, we get back a final combined image of the three channels. This is useful in circumstances when we just want to change the color of a single channel without affecting others. Below, we raise the curve of the Red channel. This in effect is adding more of the Red color to our final image.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SD_uN7j8k8jLS3jb.jpg" /><figcaption>Red Channel Curve</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*sd1yrVYyKEOgmr8T.png" /><figcaption>Original</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rh1V0CUHECi4bjxo.jpg" /><figcaption>After</figcaption></figure><h3>RGB — Red Channel</h3><p>Besides applying a Photo Curve on individual color channels (RGB) and getting back a combined image. We can aslo apply the curve on an individual channel such as the Red Channel and then get back the image of this specific channel. We can change the Channel Mode to “RGB Red” as shown below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*d1JM9JXNY9ifnEqy.jpeg" /><figcaption>Channel Mode: RGB Red</figcaption></figure><p>Working with a single channel is useful in circumstances when one of the color channels in the photo enables us to more easily extract a mask.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*AUe-iRCBmw_eBanR.jpeg" /><figcaption>Red Channel: Linear Photo Curve</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*VhiDjYvQihl6C8Js.png" /><figcaption>Original</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*9vcGQ4UDlqgKIbgh.jpg" /><figcaption>After</figcaption></figure><h3>Open Source Project</h3><p>App Store: <a href="https://apps.apple.com/us/app/photo-tone-curves/id6448858078">Photo Tone Curves app</a></p><p>Source: <a href="https://photorealityar.com/photoeditor/PhotoCurves.zip">PhotoCurves.zip</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*6isSeQ9t3S0jnPMH.png" /></figure><p><a href="https://github.com/Misfits-Rebels-Outcasts/Photo-Curves">GitHub: Photo Curves</a></p><p>If you’ve enjoyed this article and want to see more in this series follow me for updates. Also be sure to tap the 👏 button (you can do this 50 times 😱)</p><p><em>Originally published at </em><a href="https://photorealityar.com/photocurves.html"><em>https://photorealityar.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a1cc09c62e0" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Mixing and Digital Compositing Metal Shaders and CIFilter]]></title>
            <link>https://medium.com/coinmonks/mixing-and-digital-compositing-metal-shaders-and-cifilter-33e73022b25a?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/33e73022b25a</guid>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[cifilter]]></category>
            <category><![CDATA[apple-metal]]></category>
            <category><![CDATA[shaders]]></category>
            <category><![CDATA[core-image]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Fri, 24 Mar 2023 23:09:50 GMT</pubDate>
            <atom:updated>2023-06-13T07:54:16.719Z</atom:updated>
            <content:encoded><![CDATA[<h4>This article explores the fun of mixing, chaining, and digital compositing photo Images, graphic Shaders (Apple Metal Shaders), and image filters (Core Image CIFilter). Beyond the fun, we explore the advantages of surfacing the properties of shaders and filters through a Digital Compositing Pipeline, enabling an end user such as a creative graphics designer to make changes to the effect with a node graph without programming.</h4><p>For example, we explore the fun of mixing this Image</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Bxkd17zulTFAjVk0.JPG" /></figure><p>with this Shader (Fractal Flow Noise)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*EzZqhiQFwAq3W5Ht.JPG" /></figure><p>to get this Photo Fire Effects..</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*2jBSs1Xe2SZJ7k-_.gif" /></figure><h3>Fire Flame Shader Photo Effect</h3><p>To create the Fire Photo Effect, we first develop a “fire like” Shader and use a mask to create the effect of edges on the image catching fire. A Difference of Gaussian on the grayscale of the original image is first computed and then used to produce a sketch effect mask.</p><p>We need to chain several filters to create the mask and a blend with mask filter to put everything together. Our project uses Swift, Core Image CIFilter, and Apple Metal Shader to achieve all this.</p><h4>Just in case: What is a Shader?</h4><p>Image Filters is common and easily understood. Many modern software applications or apps provide image filtering capabilities. But then what is a Shader? From <a href="https://en.wikipedia.org/wiki/Shader">Wikipedia</a>, we know that “<em>a shader is a computer program that calculates the appropriate levels of light, darkness, and color during the rendering of a … process known as shading</em>”.</p><p>In other words, a Shader is a set of computer instructions (computer program) that affects the light, darkness, and color etc. of an image and it runs fast. Beyond the above, it can also perform things like animations or procedural texture image generation.</p><h3>Fractal Flow Noise Shader</h3><ul><li><a href="https://github.com/Misfits-Rebels-Outcasts/ShaderMuse/blob/main/code/ShaderMuse/Shared/Nodef/Filters/ShaderEffects/CIFiltersExtensions/FractalFlowNoiseFilter.swift">FractalFlowNoiseFilter.swift</a> (open-source)</li><li><a href="https://github.com/Misfits-Rebels-Outcasts/ShaderMuse/blob/main/code/ShaderMuse/Shared/Nodef/Filters/ShaderEffects/CIFiltersExtensions/fractalFlowNoise.metal">fractalFlowNoise.metal</a> (open-source)</li></ul><p>The Metal Shader we have developed for use in the Fire Effect is a Fractal Flow Noise Shader. This Shader is a fractal sum of a given base image. What this means is that it takes a basic image file, makes many copies of it, some zoomed, some rotated, and blends them together. The blending is not uniform, however. The images that are zoomed smaller (having a higher frequency) will be blended with a lighter weight.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*kkp6Xn9MopdM5yL2.JPG" /></figure><p>We also animate each copy of the basic image so that they move independently from each other. Furthermore, the animation of the pixels in each copy is dependent on the location of the pixel. Some pixels will move / rotate / scale faster than other pixels. This complex combination gives rise to animation effects that can simulate some natural phenomenon such as flame and gas.</p><p>Fractal Sum</p><p><em>This section is a technical discussion on the implementation of the Metal Shader with reference to the source code above.</em></p><p>This main work is performed in the ‘turbulance_p’ function. If you look at it closely, you will notice a loop that makes 24 copies of the image</p><p>for (float i= 1.0;i &lt;24.0;i++ )</p><p>For each iteration, the image is zoomed out by 1.4, while the weight is reduced by 1.5. The zooming is achieved using the pixel position represented by the variable p described in the previous section</p><p>amplitude *= 1.5; //weight pos *= 1.4; //scale</p><p>So, the final sum is something like a series as below</p><p>val = amplitude * sample(pos) + amplitude * 1/1.5 * sample(pos <em>1.4) + amplitude * 1/1.5 * 1/1.5 * sample(pos</em> 1.4*1.4) + ….</p><p>Note : The val (float3) is not just a single value. It is a 3D vector that represents the sampled value of each of the components Red, Green , Blue of the final color that is returned</p><p>Note : To zoom out the image, we simply multiply the sampling coordinates by 1.4 for each iteration. So, if we originally sample at the uv coordinates 0.0 to 1.0, the next iteration will be sampled at 0.0 to 1.4</p><h4>Animation</h4><p>To make the result even more interesting, we try to animate each copy of the basic image according to the time elapsed. The animation is a simple movement / shift of the image copy. The higher the frequency (image zoomed smaller), the more it is shifted.</p><p>The amount of shift is tracked by the variable newpos. For each iteration, the shift increases like a geometric series</p><p>newpos+= t*0.23;</p><p>In series form, the new position will take the form newpos = pos + t * 0.23 + t * t * 0.23 * 0.23 + t * t * t * 0.23 * 0.23 * 0.23 + …..</p><p>We do not sample the zoomed copy of the basic image using just the ‘newpos’ variable. Here, we introduce an additional control parameter known as ‘strength’. This ‘strength’ variable is just a weight to control the amount of shift / movement. For the current implementation, this weight is a constant factor at 0.5. There is no restriction or whatever why it cannot vary according to each iteration.</p><p>adv = (newpos — pos)*strength; pos = pos + adv;</p><p>//comment : take the average between the desired new position and current position and use the value (pos) to sample the image</p><p>Due to the computation of the ‘newpos’ variable as a linear direction, the animation will generally take the form of a main direction (with varying speeds for each copy). Now we want to make different parts of the image move differently, so we create distribution functions that vary according to the position ‘p’ of the current pixel. These distribution functions (named pdf and pdf2 for the x and y axes), form a layer that allows each pixel of each image copy to move slightly differently from its neighbors.</p><p>Example: for pdf1 and pdf2 functions float val = abs(atan(p.y <em>0.9543))</em>(cos(p.x <em>0.9532)+sinh(p.y</em>0.9816)); float val = length(p)<em>(cos(p.y/p.x</em> 0.9532)+sinh(p.y*0.9816));</p><p>The original implementation of these probability distribution functions is based approximately on polar coordinates with the length and arctangent functions. The actual implementation makes modifications to them largely by trial and error. The main intent is for these functions to be periodic , so you will notice the presence of sinusoidal and hyperbolic functions. To generate an interesting animation, these functions and their parameters are arbitrarily selected, and the reader may want to experiment by plugging different types of functions and parameters values to ‘engineer’ some other variations.</p><h3>Difference of Gaussian Sketch Mask</h3><p>With the “fire” (Fractal Flow Noise Shader), we now need to make it occur naturally in the original image. In our implementation, we use a Difference of Gaussians for creating the mask. We compute a sketch of the original image and use it as a mask to blend it with the “fire” Shader to make the edges of some parts of the image look like it is catching fire. We can use a node graph of CIFilter to illustrate how we get the mask.</p><pre>Original Image <br>| <br>V <br>CIPhotoEffectNoir <br>| <br>V <br>CIColorInvert <br>|              | <br>V              V <br>CIGaussianBlur CIGaussianBlur <br>|              | <br>V              V <br>CIColorDodgeBlend</pre><p>This gives us a sketch mask for our Fire Effect as shown below:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hHm2vZ9Fvq3usI7j.JPG" /></figure><p>We use different Radius values in the two Gaussian Blur to vary the output of our Color Dodge Blend. Below are some sample values.</p><ul><li>Gaussian Blur 1 Radius — 14.88 (or 5.18)</li><li>Gaussian Blur 2 Radius — 4.63</li></ul><pre>Original Image <br>| <br>|       CIFractalFlowNoise CIColorDodgeBlend <br>|       |                  |<br>V       V                  V<br>CIBlendWithMask</pre><p><em>Note: If you are looking at the above node graph on your phone, please tilt to landscape.</em></p><p>The CIBlendWithMask uses the Original Image as the input image, CIFractalFlowNoise as the background image, and the output of CIColorDodgeBlend (DoG sketch) as the mask.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/633/1*ilyqAm1f_QT_B2_KnMAGPw.jpeg" /></figure><h3>Fire Shader Effect — Full Node Graph</h3><pre><br>Original Image <br>|         | <br>|         V <br>|         CIPhotoEffectNoir <br>|         | <br>|         V <br>|         CIColorInvert <br>|         |              |        <br>|         V              V <br>|         CIGaussianBlur CIGaussianBlur <br>|         |              | <br>|         V              V <br>|         CIColorDodgeBlend CIFractalFlowNoise <br>|         |                 | <br>V         V                 V <br>CIBlendWithMask<br><br>  input - Original Image,<br>  background - CIFractalFlowNoise<br>  mask - CIColorDodgeBlend</pre><p><em>Note: If you are looking at the above node graph on your phone, please tilt to landscape.</em></p><p>It is possible to develop a Metal Shader to generate the entire Fire Shader Photo Effect above. But instead, we have chosen to apply Digital Compositing concepts and created a node graph of CIFilter with a considerably basic Metal Shader to achieve the same thing.</p><p><strong>The advantage of doing this is many of the programming steps is now surfaced and is controllable by a user. For example, a creative graphics designer using an app that uses a Digital Compositing node graph can easily swap in a better “fire” or “mask”. Or add additional steps to control the exposure or color correct their video.</strong></p><h3>Just in case: what is Digital Compositing?</h3><p>Digital Compositing (node-based) is the process of combining multiple seemingly simple nodes to render and achieve a desired result. The paradigm of a node-based tool involves linking basic media objects onto a procedural map or node graph and then intuitively laying out each of the steps in a sequential progression of inputs and outputs. Any parameters from an earlier step can be modified to change the outcome, with the results instantly being visible to you, and each of the nodes, being procedural, can be easily reused, saving time and effort.</p><h3>Beyond a Node Graph: Digital Compositing Pipeline</h3><p>We can also express the node graph above succinctly in English as a list of steps.</p><p>0. Original Image</p><ol><li>Photo Effect Noir (0)</li><li>Color Invert (1)</li><li>Gaussian Blur (2)</li><li>Gaussian Blur (2)</li><li>Color Dodge Blend (3,4)</li><li>Fractal Flow Noise (none)</li><li>Blend With Mask (input — 0, background — 6, mask — 5)</li></ol><p>The (0) in Step 1 refers to the input image used by the Photo Effect Noir filter. Steps 3, 4, and 5 compute the Difference of Gaussians of the grayscale image to produce a sketch effect that is further used as a mask to create the effects of edges catching fire.</p><h3>Why use a Digital Compositing Pipeline?</h3><p>Displaying a node graph as a list of steps, a <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/blob/main/documentation/NodePipeline.md">Digital Compositing Pipeline</a> (invented by Nodef), is very useful in circumstances when there is limited screen space. For example, <strong>in times when we need to view or change a node graph on a mobile device</strong>.</p><h3>ShaderMuse Open-Source Project</h3><p>We have put all the above into an open-source project: <a href="https://github.com/Misfits-Rebels-Outcasts/ShaderMuse"><strong>ShaderMuse</strong></a>. The project forked <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef">Nodef</a> to enable Digital Compositing of Shaders and Filters in iOS (iPhone, iPad, and Mac).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SDO9pS5H6VmJyToS.jpeg" /></figure><p>Check out the compiled app of Shader Muse: <a href="https://apps.apple.com/us/app/nodef-photo-filters-effects/id1640788489">Pipeline</a>.</p><h3>Adding Smoke to the Fire Effect</h3><ol><li>Download and run the Shader Muse <a href="https://apps.apple.com/us/app/nodef-photo-filters-effects/id1640788489">Pipeline</a> app -</li><li>In the Presets of the app, tap on the FIRE preset to apply the Fire Effect.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hHW0IJ3mikLNyKLL.jpeg" /></figure><ol><li>Tap on the “f” button to view the Digital Compositing Pipeline of our Fire Effect.We can further enhance this effect by adding smoke: “FBM Noise” Shader.</li><li>Next add a “Mix” Composite Filter to mix our Fire Effects with the “FBM Noise”.</li><li>Tap on the “Mix” node and change the Background to “7” (our Fire Photo Effect).</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*MvF43dXEDEWcwgD_.jpeg" /></figure><p>You can also adjust the intensity of the Fire Effect by tapping on the Gaussian Blur nodes and changing the Radius.</p><h3>Current List of Metal Shaders in Shader Muse</h3><ul><li>FBM Noise — based on 2D Noise by Morgan Mcguire @morgan3d</li><li><a href="https://github.com/Misfits-Rebels-Outcasts/ShaderMuse/blob/main/documentation/FractalFlowNoiseShader.md">Fractal Flow Noise Shader</a></li><li>Particles</li></ul><p>and over 150 CIFilter for Digital Compositing. Please clap for this article or star our <a href="https://github.com/Misfits-Rebels-Outcasts/ShaderMuse">GitHub project</a>, we will add more Shaders with your kind encouragement.</p><ul><li>Swift</li><li>Metal Shading Language</li></ul><h3>Shader Gallery</h3><p>More shaders pipeline are available in our <a href="https://github.com/Misfits-Rebels-Outcasts/ShaderMuse/blob/main/gallery/README.md">Shaders Gallery</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*o7OvD_TlfTIHZ_eiW4QaVQ.gif" /><figcaption>Sinusoidal Noise</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/482/1*MQ-BDM3O0vXfqOI4nL_K6Q.gif" /><figcaption>Lake Noise</figcaption></figure><h3>Epilogue</h3><p>After the plague, “quando nella egregia città di Fiorenza, oltre a ogn’altra italica bellissima, pervenne la mortifera pestilenza” (Decameron), while traveling, in the beautiful far east town of Malacca, a fire occurred, and both the town and skyline in front of our window was ablaze with a red orange color.</p><p>Memento Mori, Memento Vivere.</p><p><em>Originally published at </em><a href="https://github.com/Misfits-Rebels-Outcasts/ShaderMuse"><em>https://github.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33e73022b25a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/coinmonks/mixing-and-digital-compositing-metal-shaders-and-cifilter-33e73022b25a">Mixing and Digital Compositing Metal Shaders and CIFilter</a> was originally published in <a href="https://medium.com/coinmonks">Coinmonks</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Extract RGB Channels with CIColorMatrix: VFX Programming]]></title>
            <link>https://medium.com/@ConnectCode/extract-rgb-channels-with-cicolormatrix-vfx-programming-e6425fec06bb?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/e6425fec06bb</guid>
            <category><![CDATA[ios-app-development]]></category>
            <category><![CDATA[vfx-compositing]]></category>
            <category><![CDATA[swift]]></category>
            <category><![CDATA[cifilter]]></category>
            <category><![CDATA[core-image]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Thu, 22 Dec 2022 05:20:40 GMT</pubDate>
            <atom:updated>2022-12-22T05:20:40.424Z</atom:updated>
            <content:encoded><![CDATA[<h3>VFX (Visual Effects) Programming</h3><p>Visual Effects (VFX) is a process by which imagery is created, manipulated, or enhanced for filmmaking and video production. It enables filmmakers to enhance a story by bringing monsters, believable characters, worlds, special effects, and stunts to life. As a computer programmer, this is a field that greatly interests me, especially in the use of different graphical algorithms in VFX. This series is a collection of code samples that details my learning journey on VFX with Apple’s Core Image, CIFilter, and Metal.</p><h3>RGB Channels of an Image</h3><p>As I am no VFX artist, the first thing I did, as with many others, is by reading the “Digital Compositing for Film and Video” book by Steve Wright. One early thing I learned is that the foundation of any VFX is <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef">Compositing</a>, Rotoscoping, and Mattes.</p><p>Mattes can be thought of as masks in computer programming and are usually used to combine a foreground image, e.g., an actor filmed on a green screen, with a background image, e.g., a scenic mountain or a desert scene.</p><p>Mattes are a good starting point for computer programmers like me for learning VFX. And from Steve Wright’s book, I quickly learned that to work with Mattes we need to be able to extract the RGB components of a still image of a film, that is representing an image in three primary component colors: red, green, and blue. There is also an alpha channel, that indicates how opaque each pixel of an image is.</p><p>For example, the images below show the different RGB channels of the Sunflower painting of Van Gogh.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vL8arP3J2ivmZluhHVv6bQ.png" /><figcaption>RGB Channels of an Image</figcaption></figure><p>And many popular VFX or professional imaging tools display the channels as a grayscale image. In the single-channeled grayscale images below, each pixel carries only information about the intensity of light. The pixels are exclusively made up of shades of gray. This helps in simplifying algorithms and as well eliminates the complexities related to graphical computations.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*umhZRYOmJdMV6DhEZYp7Iw.png" /><figcaption>RGB Channels in Grayscale</figcaption></figure><h3>Color Matrix</h3><p>An image contains RGB channels, and we need a way to extract them. A Color Matrix does exactly that. A Color Matrix is used in computer graphics to transform colors by an arbitrary 3x3 matrix. Or a 4x4 matrix, if you include the alpha channel. The matrix below shows us a 4x4 sample.</p><pre>   |<br>   V<br><br> Input<br><br>R G B A<br>1 0 0 0 (-&gt; Red Output)<br>0 1 0 0 (-&gt; Green Output)<br>0 0 1 0 (-&gt; Blue Output)<br>0 0 0 1 (-&gt; Alpha Output)</pre><p>The way to read this is the red data of an image comes in from column 1, the green data comes in from column 2, and blue from column 3, and the alpha from column 4. The output goes out in rows, with the red output in the first row, the green output in the second row, and the blue output in the third row.</p><p>The matrix above is an Identity Color Matrix where incoming red data comes in and goes out as red, green data comes in and goes out as green, blue as blue, and alpha as alpha. So, an Identity Color Matrix does not affect the color of an image. The Color Matrix is very useful in helping us extract the different channels of an image. Below, we can specify a 0 in the matrix to turn off specific channels.</p><h3>Matrix for Extracting Red Channel</h3><pre>R G B A<br>1 0 0 0 (-&gt; Red Output)<br>0 0 0 0 (-&gt; Green Output)<br>0 0 0 0 (-&gt; Blue Output)<br>0 0 0 1 (-&gt; Alpha Output)</pre><h3>Matrix for Extracting Green Channel</h3><pre>R G B A<br>0 0 0 0 (-&gt; Red Output)<br>0 1 0 0 (-&gt; Green Output)<br>0 0 0 0 (-&gt; Blue Output)<br>0 0 0 1 (-&gt; Alpha Output)</pre><h3>Matrix for Extracting Blue Channel</h3><pre>R G B A<br>0 0 0 0 (-&gt; Red Output)<br>0 0 0 0 (-&gt; Green Output)<br>0 0 1 0 (-&gt; Blue Output)<br>0 0 0 1 (-&gt; Alpha Output)</pre><h3>Extract RGB Channels with CIColorMatrix (Core Image CIFilter)</h3><p>Finally, going into programming, Apple provides a CIColorMatrix filter, part of Core Image CIFilter. We can easily use this CIFilter and specify the necessary matrix values to extract the different channels of an image. The code below shows us how to create a CIColorMatrix filter and provide matrix input values as vectors.</p><pre>func getCIFilter(_ ciImage: CIImage)-&gt;CIFilter<br>{<br>  var currentCIFilter = CIFilter(name: &quot;CIColorMatrix&quot;)!<br>  currentCIFilter.setValue(ciImage, forKey: kCIInputImageKey)<br>  let rvector=CIVector(x:rx,y:ry,z:rz,w:rw)<br>  currentCIFilter.setValue(rvector, forKey: &quot;inputRVector&quot;)<br>  let gvector=CIVector(x:gx,y:gy,z:gz,w:gw)<br>  currentCIFilter.setValue(gvector, forKey: &quot;inputGVector&quot;)<br>  let avector=CIVector(x:ax,y:ay,z:az,w:aw)<br>  currentCIFilter.setValue(avector, forKey: &quot;inputAVector&quot;)<br>  let bvector=CIVector(x:bx,y:by,z:bz,w:bw)<br>  currentCIFilter.setValue(bvector, forKey: &quot;inputBVector&quot;)<br>  let biasvector=CIVector(x:biasx,y:biasy,z:biasz,w:biasw)<br>  currentCIFilter.setValue(biasvector, forKey: &quot;inputBiasVector&quot;)<br>  return currentCIFilter<br>}</pre><p>For the code above,</p><p>· rx is used to represent the Row 1, Column 1 value in the matrix below.</p><p>· ry is used to represent the Row 2, Column 1 value in the matrix below.</p><p>· rz is used to represent the Row 3, Column 1 value in the matrix below.</p><p>· rw is used to represent the Row 4, Column 1 value in the matrix below.</p><pre>R G B A<br>1 . . . (-&gt; Red Output)<br>0 . . . (-&gt; Green Output)<br>0 . . . (-&gt; Blue Output)<br>0 . . . (-&gt; Alpha Output)</pre><p>Thus, to extract the Red Channel, we can specify our matrix with the code below:</p><pre>//Extract Red Channel<br>colorMatrix.rx=1.0<br>colorMatrix.ry=0.0<br>colorMatrix.rz=0.0<br>colorMatrix.rw=0.0<br>colorMatrix.gx=0.0<br>colorMatrix.gy=0.0<br>colorMatrix.gz=0.0<br>colorMatrix.gw=0.0<br>colorMatrix.bx=0.0<br>colorMatrix.by=0.0<br>colorMatrix.bz=0.0<br>colorMatrix.bw=0.0<br>colorMatrix.ax=0.0<br>colorMatrix.ay=0.0<br>colorMatrix.az=0.0<br>colorMatrix.aw=1.0</pre><h3>Extract RGB Channels as a Grayscale image</h3><p>We can also display the extracted channel in Grayscale with the following:</p><pre>//Extract Red Channel<br>colorMatrix.rx=1.0<br>colorMatrix.ry=0.0<br>colorMatrix.rz=0.0<br>colorMatrix.rw=0.0<br>colorMatrix.gx=1.0<br>colorMatrix.gy=0.0<br>colorMatrix.gz=0.0<br>colorMatrix.gw=0.0<br>colorMatrix.bx=1.0<br>colorMatrix.by=0.0<br>colorMatrix.bz=0.0<br>colorMatrix.bw=0.0<br>colorMatrix.ax=0.0<br>colorMatrix.ay=0.0<br>colorMatrix.az=0.0<br>colorMatrix.aw=1.0</pre><p>In the above, we set the green and blue channels to the values of the red channel. When all three of R, G, and B are equal, we get a neutral Grayscale color: white, grey, or black.</p><h3>GitHub Sample</h3><p>The <a href="https://github.com/WebAssemblyMan/VFXProgramming">full sample is available on GitHub</a> here. By changing the matrix values, we can also extract the green or blue channels of an image. We also provide an <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef">open source</a> demo iOS app: <a href="https://apps.apple.com/us/app/nodef-photo-filters-effects/id1640788489">Nodef</a>, if you want to test out the Color Matrix filter without touching any codes.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/351/1*oghJyGhOSy1lga0627YQIw.png" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=e6425fec06bb" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Digital Compositing on Mobile]]></title>
            <link>https://medium.com/@ConnectCode/digital-compositing-on-mobile-5b3b70d49514?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/5b3b70d49514</guid>
            <category><![CDATA[open-source-software]]></category>
            <category><![CDATA[iphone-app-development]]></category>
            <category><![CDATA[photo-filters]]></category>
            <category><![CDATA[blender]]></category>
            <category><![CDATA[digital-compositing]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Tue, 06 Sep 2022 00:22:20 GMT</pubDate>
            <atom:updated>2022-09-06T00:22:20.337Z</atom:updated>
            <content:encoded><![CDATA[<p>Node-based Compositing is the process of combining multiple seemingly simple nodes to render and achieve a desired result. This process has wide applications in the graphics software industry, especially in procedural image generation, motion graphics, animation, 3D modeling, and visual effects VFX.</p><p>The paradigm of a node-based tool involves linking basic media objects onto a procedural map or node graph and then intuitively laying out each of the steps in a sequential progression of inputs and outputs. Any parameters from an earlier step can be modified to change the final outcome, with the results instantly being visible to you, and each of the nodes, being procedural, can be easily reused, saving time and effort.</p><p>To visualize what it looks like, the user interface of a node compositing software is shown below. Videos or images are first read in and then processed by a sequence of operations before being merge into one final outcome. The sequence of operations, the node graph — Directed Acyclic Graph (DAG), is represented by a flowchart.</p><pre>Read media  Read media<br> |           |     <br> V           V     <br> Premult     Color Correction<br> |           |<br> |           V<br> |           Transform<br> |           |<br> |           V<br> |           Gaussian Blur<br> |           |<br> V           V<br>     Merge<br>       |<br>       V<br>     Viewer</pre><p>As the number of nodes increases, the node graph, together with a large monitor screen, provides an overview of what is happening while also enabling one to focus on a particular progression step (node) and make changes to that step to achieve the end results.</p><h3>The Small Screen Problem on mobile</h3><p>On a mobile device, a pure node-based app is not common. Resource constraint is one limiting issue. Putting that aside, since mobile has come a long way, with mobile chips enabling one to do many advanced image and video capabilities, the other limiting issue is the small screen size of a mobile device for displaying the node-based user interface.</p><p>As a start, the arrangement of the multiple node properties screen, the node graph, the dope sheet, and the curve editor together in one small screen presents a problem. Compositing a node graph with a flowchart-like user interface is also more difficult especially when trying to connect nodes while panning the small screen frequently. This is made worst when you need to connect to a distant node with multiple inputs; or if you need to link node properties, which may involve the timeline, for tracking, animating objects, or rotoscoping.</p><h3>Main Idea</h3><p>A Node Pipeline is proposed to represent the node graph on a mobile device instead of using a flowchart-like user interface.</p><h3>Node Pipeline for Node Graph</h3><p>A node graph is commonly represented as a flowchart with a Directed Acyclic Graph (DAG) in most digital compositing software. This is shown below:</p><pre>Read media  Read media<br> |           |     <br> V           V     <br> Premult     Color Correction<br> |           |<br> |           V<br> |           Transform<br> |           |<br> |           V<br> |           Gaussian Blur<br> |           |<br> V           V<br>     Merge<br>       |<br>       V<br>     Viewer</pre><p>On a small screen mobile device, the DAG could be decomposed into multiple series of nodes shown below.</p><p>For example, the node graph above can be represented as a simple list below:</p><pre>Read media<br> |                <br> V               <br> Premult     <br></pre><pre> Read media<br> |              <br> V             <br> Color Correction<br> |           <br> V<br> Transform<br> |           <br> V          <br> Gaussian Blur      <br> <br> <br> Merge<br> |<br> V<br> Viewer</pre><p>Note the break from the first series after the ‘Premult’ node and the second break after the ‘Gaussian Blur’ node. From here, we will refer to the above arrangement as a pipeline (or Node Pipeline with breaks).</p><p>This arrangement has the obvious advantage of being represented easily on a mobile device with a simple list as shown below.</p><pre>1. Read media         &gt;<br>2. Premult            &gt;</pre><pre>3. Read media         &gt;<br>4. Color Correction   &gt;<br>5. Transform          &gt;<br>6. Blur               &gt;</pre><pre>7. Merge 2,6          &gt;<br>8. Viewer             &gt;</pre><p>The lines joining the nodes could be further represented by listing the input nodes used, like using a spreadsheet formula (e.g., =SUM(A1,A2)). In the above, the ‘Premult’ node uses node 1 (Read media) as the input, while the ‘Merge’ node uses node 2 and 6. We can also represent this information on the pipeline.</p><pre>1. Read media       (input none)  &gt;<br>2. Premult          (input 1)     &gt;</pre><pre>3. Read media       (input none)  &gt;<br>4. Color Correction (input 3)     &gt;<br>5. Transform        (input 4)     &gt;<br>6. Gaussian Blur    (input 5)     &gt;</pre><pre>7. Merge            (input 2,6)   &gt;<br>8. Viewer           (input 7)     &gt;</pre><p>Simplifying it further</p><pre>1. Read media       (none)  &gt;<br>2. Premult          (1)     &gt;</pre><pre>3. Read media       (none)  &gt;<br>4. Color Correction (3)     &gt;<br>5. Transform        (4)     &gt;<br>6. Gaussian Blur    (5)     &gt;</pre><pre>7. Merge            (2,6)   &gt;<br>8. Viewer           (7)     &gt;</pre><h3>Implications</h3><p>With the above, let’s think a little further about the implications if we use a node pipeline as the user interface for managing a node graph on a small-screen mobile device.</p><ol><li>A node pipeline will require a shift in thinking when you are using it to manage the node graph. Instead of three parallel series of nodes in the node graph above, a pipeline has three sequences (or series) of nodes. The lines joining nodes in a node graph are now represented by the target node referencing input nodes. For example, node 2 references node 1 (as the input). Node 7 references node 2 and 6.</li><li>It should be noted that, mathematically, a node graph, which is a Directed Acyclic Graph, can be decomposed to a node pipeline(the series of nodes above) without any loss of information. This means a pipeline user interface can be as flexible as a flowchart-like node graph.</li><li>A pipeline makes it easy to navigate the node graph on a small screen as you can quickly scroll up and down the list. Tapping on any item in the list (a node) can further bring up a screen for changing node properties. This user interface should be familiar to many mobile users as it is similar to the user interface for managing mobile phone ‘Settings’.</li><li>The contextual overload of a user frequently panning a flowchart-like node graph on a small screen is significantly reduced. This will improve the usability and ease of managing a node graph on a small screen.</li><li>The referencing model for linking nodes on a pipeline is similar to using a spreadsheet. ‘Merge’ below is like a formula applying to cell 2 and 6.</li><li>7. Merge (2,6) &gt;</li></ol><p>The pipeline approach leads to other interesting productivity gains in compositing on a mobile device. Please check out the following:</p><ul><li><a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/blob/main/documentation/AutoChaining.md">Auto Chaining</a> &amp; <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/blob/main/documentation/ReverseCompositing.md">Reverse Compositing</a></li><li><a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/blob/main/documentation/ViewerCycling.md">Viewer Cycling</a></li><li><a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/blob/main/documentation/DirectedAcyclicGraphGeneration.md">Directed Acyclic Graph (DAG) Generation/Import</a></li></ul><p>In the above, we will address how to join nodes by Auto Chaining and Reverse Compositing, ponder further on linking node properties, and also about setting the Viewer to a selected intermediate node on the pipeline.</p><p>You can also check out our Open Source <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef"><strong>Nodef</strong></a> project in Github.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zHXE8sp-kwYbGn5t0Rp0Qg.png" /><figcaption><a href="https://apps.apple.com/us/app/nodef-photo-filters-effects/id1640788489">Nodef app on iPhone</a></figcaption></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5b3b70d49514" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[From chaining and blending to digital compositing Core Image CIFilter]]></title>
            <link>https://medium.com/@ConnectCode/from-chaining-and-blending-to-digital-compositing-core-image-cifilter-62c308277865?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/62c308277865</guid>
            <category><![CDATA[swift-programming]]></category>
            <category><![CDATA[digital-compositing]]></category>
            <category><![CDATA[core-image]]></category>
            <category><![CDATA[image-filter]]></category>
            <category><![CDATA[cifilter]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Mon, 05 Sep 2022 23:27:31 GMT</pubDate>
            <atom:updated>2022-09-05T23:27:31.035Z</atom:updated>
            <content:encoded><![CDATA[<p>Core Image is a powerful iOS framework that makes hardware-accelerated image manipulation easy. Oftentimes, as developers, we use it to add graphical effects to images in our app. The typical process involves choosing a right CIFilter, setting parameters, and applying the filter to an image.</p><p>Sometimes, the process may involve chaining several filters to get the desired result. For example, we can use a CIColorControls filter to first adjust the saturation of an image followed by applying a Vignette filter to add a dark fading border around the edges.</p><h3>Background for a wrapper Library</h3><p>Core Image CIFilter by itself is already a full fledged library that we are able to use independently. Furthermore, we often find ourselves referring to the open-source <a href="https://github.com/FlexMonkey/Filterpedia">Filterpedia</a> project by Simon Gladman for implementation details. With these, we should already have everything that we need. So, why bother with another library?</p><p>There may be times when we need to go beyond just chaining a few CIFilter. We may need to chain filters, blend the output with another image, and then apply more filters to achieve the desired result. Or in other words, we may need to <strong>apply some kind of Node Graph (Digital Compositing) </strong>to get the effect we wanted. As developers, we often find ourselves in this programmatical chaining and blending process repeatedly.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*q0Bt5CpX71UFSkzi.JPG" /></figure><p>Beyond this, we may want to twirl the output, and then further apply an Addition composition/blend with another image. We often need to test this out in programming code.</p><p>For example, a CIEdgeWork filter produces a stylized black-and-white rendition of an image that looks similar to a woodblock cutout. The output of this filter, however, requires a background image to visualize. This requires a composite filter, CISourceAtopCompositing, to place the output of CIEdgeWork over a constant color background (CIConstantColor).</p><h3>Open Source Nodef wrapper library for Digital Compositing</h3><p>The Nodef library is a very simple wrapper for applying a node graph (of CIFilter) on an image. The library is the same library used in the open-source <a href="https://apps.apple.com/us/app/nodef-photo-filters-effects/id1640788489"><strong>Nodef</strong></a> app that that reimagines node-based compositing on a mobile device with an innovative <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/blob/main/documentation/NodePipeline.md"><strong>Node Pipeline</strong></a>. The app enables us to perform many of the compositing behavior we desire without changing a single line of code.</p><p>Besides providing a library for node-based compositing, the library also provides default values for each of the different filters and implements the ‘Codable’ protocol for saving and loading a node graph in JSON. Using the JSON file, we can be creatively compositing on a mobile device, saving the composite as a file and then loading it on our desktop computer with an application or command line tool.</p><p>Happy Compositing on Mobile!</p><h3><strong>Original Image</strong></h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*WoYs8Ijr28hxnuRi.JPG" /></figure><h3><strong>Add a CIColorMonoChrome Filter</strong></h3><pre>func createOneFilter(_ inputImage: UIImage) -&gt; UIImage {</pre><pre>let filters = FiltersX() <br>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Color Monochrome&quot;)) <br>return filters.applyFilters(image: inputImage)</pre><pre>} </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*fz1ISyG9oFCFtZ60.JPG" /></figure><h3>Chain Filters</h3><p>CISepiaTone takes the original image as the input image and CIZoomBlur takes the output of CISepiaTone as the input image.</p><pre>func chainFilters(_ inputImage: UIImage) -&gt; UIImage { </pre><pre>let filters = FiltersX() <br>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Sepia Tone&quot;)) filters.add(filterHolder: filters.getFilterWithHolder(&quot;Zoom Blur&quot;)) return filters.applyFilters(image: inputImage) </pre><pre>} </pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*JPtD7bX5wyLjA4FqOWZt-Q.jpeg" /></figure><h3>Blend Filters</h3><p>First apply a CIDotScreen filter.</p><pre>func blendFilters(_ inputImage: UIImage) -&gt; UIImage { </pre><pre>let filters = FiltersX() </pre><pre>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Dot Screen&quot;)) <br>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Subtract Blend Mode&quot;)) </pre><pre>return filters.applyFilters(image: inputImage) </pre><pre>} </pre><p>CISubtractBlendMode takes the output of CIDotScreen as the inputImage and the original image as the backgroundImage.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0JfmGF0Fo5uFCTLiYJvtLw.jpeg" /></figure><h3>Create a CICheckboardGenerator filter</h3><p>CICheckboardGenerator requires no input Image.</p><pre>func generatorFilters(_ inputImage: UIImage) -&gt; UIImage { </pre><pre>let filters = FiltersX() filters.size=CGSize(width:inputImage.size.width, height:inputImage.size.height)<br> <br>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Checkerboard Generator&quot;)) </pre><pre>return filters.applyFilters(image: inputImage) </pre><pre>}</pre><p>Changing the width of CICheckboardGenerator.</p><pre>func filterProperties(_ inputImage: UIImage) -&gt; UIImage { </pre><pre>let filters = FiltersX() filters.size=CGSize(width:inputImage.size.width, height:inputImage.size.height) </pre><pre>let fxHolder=filters.getFilterWithHolder(&quot;Checkerboard Generator&quot;) (fxHolder.filter as! CheckerboardGeneratorFX).width = 500 filters.add(filterHolder: fxHolder) </pre><pre>return filters.applyFilters(image: inputImage) }</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yOUcIldj5S-86EMJcNrxmw.jpeg" /></figure><h3>Digital Compositing Filters</h3><p>Apply a CIMultiplyBlendMode on the CIColorMonochrome version of the original image with a CICheckboardGenerator.</p><pre>func compositingFilters(_ inputImage: UIImage) -&gt; UIImage { </pre><pre>let filters = FiltersX() filters.size=CGSize(width:inputImage.size.width, height:inputImage.size.height) </pre><pre>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Color Monochrome&quot;)) //Node 1 <br>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Checkerboard Generator&quot;)) //Node 2 </pre><pre>let fxHolder=filters.getFilterWithHolder(&quot;Multiply Blend Mode&quot;) (fxHolder.filter as! MultiplyBlendModeFX).inputImageAlias = &quot;2&quot; (fxHolder.filter as! MultiplyBlendModeFX).backgroundImageAlias = &quot;1&quot; filters.add(filterHolder: fxHolder) </pre><pre>return filters.applyFilters(image: inputImage) </pre><pre>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*maqSSGV3NrO2Zrc1cjhZcw.jpeg" /></figure><h3>Digital Compositing with a Node Graph</h3><p>First, chain the original image with the following filters.</p><pre>Original Image -&gt; CILineScreen -&gt; CIColorMonochrome</pre><p>Next, generate a checkboard and then apply a triangle tile.</p><pre>CheckboardGenerator -&gt; CITriangleTile</pre><p>Finally, multiply blend the output of the color monochrome with the output of the triangle tile.</p><pre>CIMultiplyBlendMode on CIColorMonochrome and CITriangleTile</pre><p>The Swift code for the node graph above.</p><pre>func nodeGraphFilters(_ inputImage: UIImage) -&gt; UIImage { </pre><pre>let filters = FiltersX() filters.size=CGSize(width:inputImage.size.width, height:inputImage.size.height) </pre><pre>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Line Screen&quot;)) //Node 1 <br>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Color Monochrome&quot;)) //Node 2 </pre><pre>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Checkerboard Generator&quot;)) //Node 3 <br>filters.add(filterHolder: filters.getFilterWithHolder(&quot;Triangle Tile&quot;)) //Node 4 </pre><pre>let fxHolder=filters.getFilterWithHolder(&quot;Multiply Blend Mode&quot;) (fxHolder.filter as! MultiplyBlendModeFX).inputImageAlias = &quot;4&quot; (fxHolder.filter as! MultiplyBlendModeFX).backgroundImageAlias = &quot;2&quot; filters.add(filterHolder: fxHolder) </pre><pre>return filters.applyFilters(image: inputImage) </pre><pre>}</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*xFiksxvlt8bMy0uU7xhEpg.jpeg" /></figure><h3>Compiling the Source</h3><ul><li>Download the <a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/tree/main/code/Nodef%20Filters%20Library">Source Code</a></li><li>Launch XCode and load Nodef.xcodeproj Build and run on iPhone Simulator or Device</li><li>or download the <a href="https://apps.apple.com/us/app/nodef-photo-filters-effects/id1640788489"><strong>Nodef</strong></a> app on the App Store</li></ul><p><em>Originally published at </em><a href="https://github.com/Misfits-Rebels-Outcasts/Nodef/blob/main/documentation/ChainingBlendingCompositingCoreImageCIFilters.md"><em>https://github.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=62c308277865" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Develop a WYSIWYG drawing app with SwiftUI]]></title>
            <link>https://medium.com/@ConnectCode/develop-a-wysiwyg-drawing-app-with-swiftui-b08477cbb2ef?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/b08477cbb2ef</guid>
            <category><![CDATA[open-source]]></category>
            <category><![CDATA[barcode-label]]></category>
            <category><![CDATA[drawing-app]]></category>
            <category><![CDATA[wysiwyg]]></category>
            <category><![CDATA[swiftui]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Tue, 26 Apr 2022 01:13:29 GMT</pubDate>
            <atom:updated>2022-04-26T01:13:29.258Z</atom:updated>
            <content:encoded><![CDATA[<p>This article explores the use of SwiftUI for developing a What-You-See-Is-What-You-Get (WYSIWYG) vector drawing app. A WYSIWYG vector drawing app can be thought of as one that renders different objects such as a rectangle, an ellipse, a text, or other shapes on a canvas; the moving of objects around a canvas by dragging; and changing object properties by tapping on objects. This is illustrated in the screenshot of an iOS app below. Specifically, this article explores the use of a <strong>SwiftUI View</strong> as the “Drawing Canvas” instead of using a Core Graphics Canvas or a SwiftUI Canvas.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*DD6C_RZjlxjmwaFk.png" /><figcaption>SwiftUI WYSIWYG</figcaption></figure><h4>Why use SwiftUI View as the drawing canvas?</h4><p>When developing a vector drawing app in the Apple ecosystem, things that come immediately onto the mind are Core Graphics or SwiftUI Canvas. Both are extremely fast, easy to use, and provide a canvas for us to draw on. Naturally, both are good choices as the canvas of a drawing app. However, when the drawing app requires WYSIWYG behavior, a Swift developer realizes that handling of object interactions such as drag, move, and resize, requires the use of Apple’s Gestures and Events.</p><p>This makes it hard for a Swift developer to ignore the use of a SwiftUI View as the “Drawing Canvas”. This is because all Apple’s Gestures and Events are directly supported by a SwiftUI View. If one is to take a step back and think about it, a SwiftUI View is designed by Apple for User Interface (UI) development, and such a View already naturally supports all the behavior required by a WYSIWYG app: rendering views and objects, and supporting gestures and events. The use of a SwiftUI View as the “Drawing Canvas” also does not prevent us from using Core Graphics, SwiftUI Canvas, or even Metal for rendering the underlying object that requires special treatment, as all three can be represented as a SwiftUI View easily.</p><h4>The outcome of the Exploration</h4><p><strong>Open Source</strong></p><p><a href="https://github.com/Misfits-Rebels-Outcasts/SwiftUI-WYSIWYG-Draw">https://github.com/Misfits-Rebels-Outcasts/SwiftUI-WYSIWYG-Draw</a></p><p><strong>App</strong></p><p><a href="https://apps.apple.com/us/app/barcode-label/id1620797490">https://apps.apple.com/us/app/barcode-label/id1620797490</a></p><p>The following section highlights the advantages, disadvantages, and challenges of using SwiftUI, with a SwiftUI View as the “Drawing Canvas”, for developing a WYSIWYG app. This is probably not a suitable design for everyone, but some points highlighted may be useful consideration to a WYSIWYG drawing app developer.</p><h4>The Design</h4><p>As mentioned above, we will be using a SwiftUI View as the “Drawing Canvas” to draw shapes, text, or images. But how do we go about doing this? The following is as simple as it gets.</p><pre><br>var body: some View { <br>  ZStack { <br>     ForEach(shapes.shapeList){ <br>                 shape in ShapeViewX() <br>     } <br>  } <br>}</pre><p>The fundamental concept is to just draw a list of shapes (represented as SwiftUI View) on a ZStack inside a SwiftUI View. The list of shapes and shape classes are defined below:</p><pre>class ShapesX{ </pre><pre>@Published var shapeList = [ShapeX]() </pre><pre>} </pre><pre>class ShapeX { </pre><pre>@Published var location: CGPoint = CGPoint(x: 50, y: 50) <br>@Published var size: CGSize = CGSize(width: 100, height: 100) @Published var canvasSize: CGSize = CGSize(width: 500, height: 500) </pre><pre>}</pre><p>Note — The code above is simplified to help convey the concepts more easily. For example, the “ObservableObject” attributes are removed.</p><p>ShapesX and ShapeX can be considered as the model objects of our vector drawings. We render them in a SwiftUI View by using a SwiftUI View to represent each shape. ShapeViewX below is used for this purpose. It is interesting to note that ShapeViewX can also be implemented with a SwiftUI Canvas, a Core Image Canvas, or with Metal. For our scenario, we will use SwiftUI shapes such as Rectangle, Ellipse, or wrap a SwiftUI View around a SwiftUI Text for representing objects on the canvas.</p><pre>struct ShapeViewX: View { </pre><pre>@ObservedObject var shape: ShapeX </pre><pre>//Handle the interactions such as Move and Resize <br>//Draw a Bounding Box <br>//Draw the Shape </pre><pre>var body: some View { </pre><pre>  self.shape.view() <br>  //depending of the type shape is, we can draw a rectangle or ellipse etc. </pre><pre>} </pre><pre>}</pre><p>ShapeViewX is used to render different types of shapes such as a rectangle or an ellipse. The view function for drawing a Rectangle is illustrated below.</p><pre>class RectangleX: ShapeX { </pre><pre>@Published var strokeWidth:Double = 1 <br>@Published var strokeColor = Color.black <br>@Published var fillColor = Color.white </pre><pre>override func view() -&gt; AnyView { </pre><pre>AnyView( <br>    Rectangle() <br>     .stroke(strokeColor, lineWidth: strokeWidth)<br>     .background(Rectangle().fill(fillColor)) <br>     .frame(width: self.size.width, height: self.size.height)<br>     .position(self.location) ) <br>} </pre><pre>}</pre><p>Besides drawing shapes, ShapeViewX can be used to easily manage all gestures and events in a single place. This is illustrated below.</p><h4>Gestures</h4><p>The Pan (drag to move) gesture for ShapeViewX can be as simple as follow:</p><pre>struct ShapeViewX: View { </pre><pre>@ObservedObject var shape: ShapeX <br>@State private var location: CGPoint = CGPoint(x: 50, y: 50) </pre><pre>var drag: some Gesture { <br>     DragGesture()<br>      .onChanged { <br>        value in self.shape.location = value.location <br>      } <br>} </pre><pre>var body: some View { </pre><pre>  ZStack { <br>    self.shape.view() </pre><pre>    Rectangle() <br>     .fill(Color.white) <br>     .opacity(0.001) <br>     .frame(width: self.shape.size.width, height:   self.shape.size.height) <br>     .position(self.shape.location) <br>     .gesture( drag ) <br>   } <br>} </pre><pre>}</pre><p>If you test the code above, you will see a little jump when we start dragging the rectangle. The following article explains this problem well and provides a solution to this.</p><p><a href="https://sarunw.com/posts/move-view-around-with-drag-gesture-in-swiftui/">https://sarunw.com/posts/move-view-around-with-drag-gesture-in-swiftui</a></p><h4>The Printing Problem</h4><p>The use of a SwiftUI View as a WYSIWYG is great for handling events and gestures. But for many vector drawing apps, the need to export, print, and/or generate a PDF output are also important. If we need to generate a PDF, we can implement something like the following:</p><h4>A Manual Solution</h4><pre>let pageRect = CGRect(x:0, y:0, width: 8.5*72, height: 11.0*72) </pre><pre><br>let renderer = UIGraphicsPDFRenderer(bounds:pageRect , format: format) </pre><pre>let pdf = renderer.pdfData { (context) in </pre><pre>context.beginPage() </pre><pre>let attributes = [ NSFontAttributeName : UIFont.boldSystemFont(ofSize: 150) ] </pre><pre>let text = &quot;Hello!&quot; as NSString </pre><pre>text.draw(in: CGRect(x: 0, y: 0, width: 500, height: 200), withAttributes: attributes) </pre><pre>}</pre><p>In the above, we draw a text (“Hello”) in the PDF. We can imagine that we need to translate each of our shapes and text into PDF with codes similar to the above. This is useful if a fine-grain control of every single rendering detail is required. It can, however, be tedious.</p><h4>Convert a SwiftUI View to PDF</h4><p>Alternatively, we can convert a SwiftUI View to PDF. This is a quick and easy way if we do not want to manually translate each shape into PDF.</p><p>See the following for more information:</p><p><a href="https://stackoverflow.com/questions/60753436/convert-swiftui-view-to-pdf-on-ios?msclkid=ead66797becb11ec9c47d91ad8634470">https://stackoverflow.com/questions/60753436/convert-swiftui-view-to-pdf-on-ios?msclkid=ead66797becb11ec9c47d91ad8634470</a></p><p>The code below illustrates how to convert a SwiftUI View and its children’s objects to PDF by using UIGraphicsPDFRenderer.</p><pre>func exportToPDF() { </pre><pre>let documentDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first! </pre><pre>let outputFileURL = documentDirectory.appendingPathComponent(&quot;SwiftUI.pdf&quot;) </pre><pre>let width: CGFloat = 8.5 * 72.0 <br>let height: CGFloat = 11.0 * 72.0 <br>let printout = SwiftUICanvasView() </pre><pre>let pdfVC = UIHostingController(rootView: printout) <br>pdfVC.view.frame = CGRect(x: 0, y: 0, width: width, height: height) </pre><pre>//Render the view behind all other views <br>let rootVC = UIApplication.shared.windows.first?.rootViewController rootVC?.addChild(pdfVC) <br>rootVC?.view.insertSubview(pdfVC.view, at: 0) </pre><pre>//Render the PDF <br>let pdfRenderer = UIGraphicsPDFRenderer(bounds: CGRect(x: 0, y: 0, width: 8.5 * 72.0, height: height)) </pre><pre>do { </pre><pre>    try pdfRenderer.writePDF(to: outputFileURL, withActions: { (context) in </pre><pre>       context.beginPage() <br>       pdfVC.view.layer.render(in: context.cgContext) <br>    }) </pre><pre>    self.exportURL = outputFileURL <br>    self.showExportSheet = true <br>}</pre><pre>catch { </pre><pre>    self.showError = true <br>    print(&quot;Could not create PDF file: \(error)&quot;) </pre><pre>} </pre><pre>pdfVC.removeFromParent() <br>pdfVC.view.removeFromSuperview() </pre><pre>}</pre><h4>The Problem with converting a SwiftUI View to PDF</h4><p>If you look at the conversation thread more closely, you will notice the following:</p><pre><em>&quot;When I tried to generate a PDF file using the the solutions in other answers I only got a blurred PDF and the quality was far from good.&quot;</em> pawello2222</pre><p>When a SwiftUI View is converted into a PDF, it is converted at 72 Dots-Per-Inch (DPI). If your drawing contains high-resolution objects such as images, they will become blurry in the PDF. A low-resolution output for a WYSIWYG graphics app is not ideal. If we cannot accept a lower resolution output, does it mean we are back to the manual method of converting our SwiftUI View to a PDF?</p><pre><em>&quot;I ended up generating the SwiftUI view in a much bigger frame and scaled down the context to the appropriate size.&quot;</em> pawello2222</pre><h4>The Possible Solution</h4><p>One solution could be to render the “Canvas” (SwiftUI View) at a required DPI (e.g. 300 DPI) and then scale it down with “scaleEffect” for viewing on the app. For example, we can render an 8.5 x 11.0 (inches) canvas at 300 DPI. To view it on the app, we can scale it down by a factor of 72.0/300.0 (72 DPI is the DPI used by an iOS app) as shown below.</p><pre><br>SwiftUICanvasView() //at 300 DPI <br>.frame(width: 8.5*300, height: 11.0*300, alignment: .center) .scaleEffect(72.0/300.0)<br></pre><p>When rendering SwiftUICanvasView, at 300 DPI, to a PDF, our PDF and its objects will no longer be blurry.</p><h4>The Additional Complexity</h4><p>The design of rendering a canvas at a higher DPI and then scaling it down for viewing will undoubtedly increase the app’s resource requirements. There will also be additional code development complexity. For example, a text that used to be rendered at a font size of 16 (in 72 DPI), when rendered on a 300 DPI canvas, will require a font size of 16 * 300.0/72.0. This translation of font size will need to be managed. The code below shows how to:</p><pre>class TextX: ShapeX { </pre><pre>//@Published var fontSize:CGFloat = 16.0 <br>@Published var fontSize:CGFloat = 16.0*300.0/72.0 </pre><pre>}</pre><h4>What have we done?</h4><p>In summary, we detailed the use of a SwiftUI View as a drawing canvas for rendering shapes and text and handling of gestures and events. We drew the SwiftUI View in 300 DPI and used “scaleEffect” to scale it down for viewing on the app. We then used the SwiftUI View (at 300 DPI) and converted it to a 72 PDF. We also highlighted the additional complexity of doing so.</p><p>Is the above design good for everyone developing a WYSIWYG app? Definitely not! But I hope, at least some parts are useful for anyone considering the development of a WYSIWYG app with SwiftUI. For us, we have used it to develop a label design WYSIWYG and barcode printing app.</p><p><em>sunt lacrimae rerum et mentem mortalia tangunt — aeneas</em></p><p><em>Originally published at </em><a href="https://www.barcoderesource.com/swiftui_view_vector_draw_wysiwyg.shtml"><em>https://www.barcoderesource.com</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b08477cbb2ef" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[.NET MAUI with .NET CLI (Command Line Interface).]]></title>
            <link>https://medium.com/@ConnectCode/net-maui-with-net-cli-command-line-interface-33543b4bcbb2?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/33543b4bcbb2</guid>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Fri, 21 Jan 2022 23:19:19 GMT</pubDate>
            <atom:updated>2022-01-21T23:19:19.982Z</atom:updated>
            <content:encoded><![CDATA[<h3>.NET MAUI with .NET CLI (Command Line Interface).</h3><p>.NET MAUI with .NET CLI (Command Line Interface)</p><p>This article describes how to develop a .NET MAUI app with the .NET CLI (Command Line Interface).</p><p>.NET Multi-platform App UI (.NET MAUI) is a cross-platform framework for creating native mobile and desktop apps with C# and XAML that can run on Android, iOS, macOS, and Windows from a single shared code-base.</p><p>1. Install .NET SDK.</p><p><a href="https://dotnet.microsoft.com/en-us/download">https://dotnet.microsoft.com/en-us/download</a></p><p>2. Install .NET MAUI workload with the dotnet CLI. A .NET “workload” is a collection of packs. Launch a command prompt and enter the following:</p><p><strong>dotnet workload install maui</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*mJo0YmYli_Fl-VGf.png" /></figure><p>Check that the MAUI workload installs successfully.</p><p><strong>dotnet workload list</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*czVqRyEma-pXeGXv.png" /></figure><p>3. Verify and install missing components with maui-check command line utility.</p><p><strong>dotnet tool install -g redth.net.MAUI.check</strong></p><p>4. Create a new folder and a new MAUI app.</p><p><strong>mkdir MyMauiApp</strong></p><p><strong>cd MyMauiApp</strong></p><p><strong>dotnet new maui</strong></p><p>5. The “dotnet new maui” command generates a template for a .NET MAUI app with all the necessary project, C#, and XAML files. You can use any editor such as Notepad or <a href="https://code.visualstudio.com/Download">Visual Studio Code</a> to edit the files.</p><p>If you are using Visual Studio Code, enter the following command in the Command Prompt to view the source code generated in the “MauiQRCodeApp” folder.</p><p>6. Start an Android Emulator with the following command:</p><pre>&quot;C:\Program Files (x86)\Android\android-sdk\emulator\emulator.EXE&quot; -no-boot-anim -avd pixel_5_-_api_30 -prop monodroid.avdname=pixel_5_-_api_30</pre><p>7. Run the MAUI app in an Android Simulator.</p><p><strong>dotnet build -t:Run -f net6.0-android</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/802/0*-B8EhzwvuyjGCoMv.png" /></figure><p><em>Originally published at </em><a href="https://mauiman.dev/maui_cli_commandlineinterface.html"><em>https://mauiman.dev</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=33543b4bcbb2" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Getting started with .NET MAUI (iOS) on an M1 Mac.]]></title>
            <link>https://medium.com/@ConnectCode/getting-started-with-net-maui-ios-on-an-m1-mac-752744422c84?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/752744422c84</guid>
            <category><![CDATA[ios]]></category>
            <category><![CDATA[dotnet]]></category>
            <category><![CDATA[c-sharp-programming]]></category>
            <category><![CDATA[maui]]></category>
            <category><![CDATA[xamarin]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Fri, 21 Jan 2022 04:57:56 GMT</pubDate>
            <atom:updated>2022-01-21T04:57:56.033Z</atom:updated>
            <content:encoded><![CDATA[<h3>Getting started with .NET MAUI (iOS) on an M1 Mac.</h3><p>Getting started with .NET MAUI (iOS) on an M1 Mac</p><p>The easiest way of getting started with .NET MAUI development (iOS) on an M1 Mac.</p><p>.NET Multi-platform App UI (.NET MAUI) is a cross-platform framework for creating native mobile and desktop apps with C# and XAML that can run on Android, iOS, macOS, and Windows from a single shared code-base.</p><p>1. Install macOS Arm64 .NET SDK on M1 Mac.</p><p><a href="https://dotnet.microsoft.com/en-us/download">https://dotnet.microsoft.com/en-us/download</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Aj-ykZLnxY0CyH9o.png" /></figure><p>Alternatively you can also install Visual Studio 2022 for Mac preview.</p><p>2. Install .NET MAUI workload with the dotnet CLI. A .NET “workload” is a collection of packs. Launch “Terminal” and enter the following:</p><p><strong>dotnet workload install maui</strong></p><p>Note — You may need “sudo” in front of the command to enable the necessary security privileges.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*v625ojS5YPlZjRTd.png" /></figure><p>Check that the MAUI workload is installed successfully.</p><p><strong>dotnet workload list</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*wWU5U2zh0_B2wOVL.png" /></figure><p>3. Create a new folder and a new MAUI app.</p><p><strong>mkdir MyMauiApp</strong></p><p><strong>cd MyMauiApp</strong></p><p><strong>dotnet new maui</strong></p><p>4. Run the MAUI app in an iOS Simulator.</p><p><strong>dotnet build -t:Run -f net6.0-ios</strong></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*3CMIGCWSZFVhr94w.png" /></figure><p>5. Install Visual Studio Code for Mac.</p><p><a href="https://code.visualstudio.com/Download">https://code.visualstudio.com/Download</a></p><p>6. Launch Visual Studio Code and open the “MyMauiApp” folder to view the source code.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*7uIsbCu_Zl6muUWA.png" /></figure><p><em>Originally published at </em><a href="https://www.mauiman.dev/maui_ios_mac.html"><em>https://www.mauiman.dev</em></a><em>.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=752744422c84" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Rust WebAssembly Hello World — Deep Dive with wasm-bindgen and wasm2wat]]></title>
            <link>https://medium.com/@ConnectCode/rust-webassembly-hello-world-deep-dive-with-wasm-bindgen-and-wasm2wat-a52270009034?source=rss-8d47150825f7------2</link>
            <guid isPermaLink="false">https://medium.com/p/a52270009034</guid>
            <category><![CDATA[webassembly]]></category>
            <category><![CDATA[rust]]></category>
            <category><![CDATA[rustlang]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[nicole]]></dc:creator>
            <pubDate>Tue, 02 Jul 2019 02:54:49 GMT</pubDate>
            <atom:updated>2019-07-02T02:54:49.747Z</atom:updated>
            <content:encoded><![CDATA[<h3>Rust WebAssembly Hello World — Deep Dive with wasm-bindgen and wasm2wat</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Wut8Ske0RF9pL7h9YP9lLg.jpeg" /></figure><p>This post first appeared on <a href="https://www.webassemblyman.com/rust_webassembly_hello_world.html"><strong>WebAssemblyMan.com: Rust WebAssembly Hello World</strong></a>.</p><p>In this article, we are going to create a WebAssembly Hello World program with Rust. You might be thinking that a WebAssembly Hello World tutorial could be found almost everywhere, including one on the Mozilla website and rustwasm.github.io. So why would you need to read this article?</p><p>This article is different as we are going beyond just getting the Hello World program to work. We will dig deeper to understand the codes generated by <strong>wasm-bindgen</strong> and how they all come together to help us with our development. The <strong>WebAssembly Binary Toolkit</strong> will also be used to explore the generated wasm code. This will give us a good understanding of Rust WebAssembly and a good foundation to start with. It also quickly becomes clear why Rust WebAssembly applications are smaller and more optimized as compared to other frameworks with Garbage Collector, and why it is a winner for developers with existing JavaScript web applications or websites. Things like <strong>web-sys</strong> (Raw API bindings for Web APIS) and <strong>WASI </strong>(WebAssembly System Interface) also become easier to appreciate.</p><p><strong><em>Prerequisites</em></strong></p><ul><li>Rust is installed — See <a href="https://www.rust-lang.org/tools/install">https://www.rust-lang.org/tools/install</a></li><li>Webpack is installed</li><li>WebAssembly Binary Toolkit (wabt) is installed</li></ul><h3>Hello World Tutorial in Rust WebAssembly</h3><p>1. Install wasm-bindgen. There are other ways to create a Hello World program without wasm-bindgen. But in this tutorial, we are going to use it as it is essential in Rust WebAssembly development.</p><p>cargo install wasm-bindgen-cli</p><p>Rust WebAssembly enables you to surgically insert WebAssembly modules into your existing JavaScript applications, especially into your performance critical code paths. You can think of wasm-bindgen as a tool that helps you do that by generating the glue and bindings for the efficient interactions between JavaScript and WebAssembly.</p><p>2. Next, we will like to create a new Rust WebAssembly project. Enter the following command</p><pre>cargo new helloworld — lib</pre><p>Please note the double dash before “lib”.</p><p>You should see the following message. A new helloworld package with your project files has been created.</p><p>Created library `helloworld` package</p><p>3. Using a text editor, open Cargo.toml. You should see the following</p><pre>[package]<br>name = &quot;helloworld&quot;<br>version = &quot;0.1.0&quot;<br>authors = [&quot;djembe-waka &quot;]<br>edition = &quot;2018&quot;</pre><pre>[dependencies]</pre><p>Change it to the following:</p><pre>[package]<br>name = &quot;helloworld&quot;<br>version = &quot;0.1.0&quot;<br>authors = [&quot;djembe-waka &quot;]<br>edition = &quot;2018&quot;</pre><pre>[lib]<br>crate-type = [&quot;cdylib&quot;]</pre><pre>[dependencies]<br>wasm-bindgen = &quot;0.2&quot;</pre><p>The “cdylib” indicator makes this Rust project expose a C-style dynamic library that is used by linkers to produce our WebAssembly module. As described above, wasm-bindgen enables or facilitates high-level interactions between wasm modules and JavaScript.</p><p>4. Using a text editor, open “src/lib.rs”. Change it to the following:</p><pre><br>extern crate wasm_bindgen; <br>use wasm_bindgen::prelude::*; </pre><pre>// Import &#39;window.alert&#39; <br>#[wasm_bindgen] <br>extern &quot;C&quot; { <br>	fn alert(s: &amp;str); <br>} </pre><pre>// Export a &#39;helloworld&#39; function <br>#[wasm_bindgen] <br>pub fn helloworld(name: &amp;str) { <br>	alert(&amp;format!(&quot;Hello World : {}!&quot;, name)); <br>}</pre><p>In the above code, wasm_bindgen allows us to import JavaScript things (‘alert’) into Rust and export Rust things (‘helloworld’) to JavaScript.</p><p>5. Compile the code. In your command line enter the following:</p><p>cargo build — target wasm32-unknown-unknown</p><p>Please note the double dash before “target”. If “wasm32-unknown-unknown target” is not installed, enter the following to install it.</p><p>rustup target add wasm32-unknown-unknown</p><p>6. Setup a web server to run our program.</p><p>We need a web server to test our WebAssembly “Hello World” program. We are going to use Webpack but you can also use another tool or web server that you are comfortable with. We need to create 3 files: index.js, package.json, and webpack.config.js. However, we will not be going into the details of these files as we are going to focus on the codes generated by wasm-bindgen.</p><p>index.js</p><pre>const rust = import(&#39;./pkg/helloworld&#39;);</pre><pre>rust<br>  .then(m =&gt; m.helloworld(&#39;World!&#39;))<br>  .catch(console.error);</pre><p>package.json</p><pre>{<br>  &quot;scripts&quot;: {<br>    &quot;build&quot;: &quot;webpack&quot;,<br>    &quot;serve&quot;: &quot;webpack-dev-server&quot;<br>  },</pre><pre>  &quot;devDependencies&quot;: {<br>    &quot;@wasm-tool/wasm-pack-plugin&quot;: &quot;0.4.2&quot;,<br>    &quot;text-encoding&quot;: &quot;^0.7.0&quot;,<br>    &quot;html-webpack-plugin&quot;: &quot;^3.2.0&quot;,<br>    &quot;webpack&quot;: &quot;^4.29.4&quot;,<br>    &quot;webpack-cli&quot;: &quot;^3.1.1&quot;,<br>    &quot;webpack-dev-server&quot;: &quot;^3.1.0&quot;<br>  }<br>}</pre><p>webpack.config.js</p><pre>const path = require(&#39;path&#39;);<br>const HtmlWebpackPlugin = require(&#39;html-webpack-plugin&#39;);<br>const webpack = require(&#39;webpack&#39;);<br>const WasmPackPlugin = require(&quot;@wasm-tool/wasm-pack-plugin&quot;);</pre><pre>module.exports = {<br>    entry: &#39;./index.js&#39;,<br>    output: {<br>        path: path.resolve(__dirname, &#39;dist&#39;),<br>        filename: &#39;index.js&#39;,<br>    },<br>    plugins: [<br>        new HtmlWebpackPlugin(),<br>        new WasmPackPlugin({<br>            crateDirectory: path.resolve(__dirname, &quot;.&quot;)<br>        }),<br>        // Have this example work in Edge which doesn&#39;t ship `TextEncoder` or<br>        // `TextDecoder` at this time.<br>        new webpack.ProvidePlugin({<br>          TextDecoder: [&#39;text-encoding&#39;, &#39;TextDecoder&#39;],<br>          TextEncoder: [&#39;text-encoding&#39;, &#39;TextEncoder&#39;]<br>        })<br>    ],<br>    mode: &#39;development&#39;<br>};</pre><p>You may require the following commands if some of the packages are not available on your system.</p><ul><li>npm install webpack — save-dev</li><li>npm install webpack-cli — save-dev</li><li>npm install webpack-dev-server — save-dev</li><li>npm install html-webpack-plugin — save-dev</li><li>npm install @wasm-tool/wasm-pack-plugin — save-dev</li><li>npm install text-encoding — save-dev</li></ul><p>Please note the double dash before “save-dev”.</p><p>7. Build the program by running the following command:</p><p>npm run build</p><p>8. Run the Hello World program</p><p>npm run serve</p><p>Navigate your browser to “localhost:8080” and see a “Hello World!” alert is displayed.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*s0gjs7PftLvCXDP8.png" /><figcaption>Rust WebAssembly Hello World</figcaption></figure><p>8. Now that we got the Hello World program running, we want to dig deeper. Go to the “pkg” folder and see that the following files are automatically generated for you during the build process.</p><ul><li>“helloworld_bg.wasm”</li><li>“helloworld.js”</li><li>“helloworld.d.ts”</li><li>“package.json”</li></ul><p>The files can also be generated manually by using the following wasm-bindgen cli command.</p><p>wasm-bindgen target/wasm32-unknown-unknown/debug/helloworld.wasm — out-dir ./pkg</p><p>Please note the double dash before “out-dir”.</p><p>9. The first thing we should do is to understand what happens when we run the Hello World program in a browser. The following shows the sequence of function calls (or more specifically the files containing the functions) that occur when we navigate point to “localhost:8080” in the browser.</p><p><strong>index.js -&gt; helloworld.js -&gt; helloworld_bg.wasm</strong></p><p>index.js</p><pre>const rust = import(&#39;./pkg/helloworld&#39;);<br>rust<br>  .then(m =&gt; m.helloworld(&#39;World!&#39;))<br>  .catch(console.error);</pre><p>“index.js” imports a “helloworld.js” package and calls the “helloworld” function in the package.</p><p>helloworld.js</p><pre><br>export function helloworld(name) {<br>    const ptr0 = passStringToWasm(name);<br>    const len0 = WASM_VECTOR_LEN;<br>    try {<br>        return wasm.helloworld(ptr0, len0);<br><br>    } finally {<br>        wasm.__wbindgen_free(ptr0, len0 * 1);<br><br>    }<br>}<br></pre><p>“helloworld.js” file is automatically generated by wasm-bindgen and contains the JavaScript glue for importing DOM and JavaScript functions into Rust. It also exposes an API on the generated WebAssembly functions to JavaScript.</p><p>Rust WebAssembly focuses on integrating WebAssembly with existing JavaScript applications. To do this, we need to pass different values, objects or structs from JavaScript in and out of WebAssembly functions. This is not easy as the different object types of two different systems need to be reconciled. To make matters worst, WebAssembly currently only supports integer and floating point numbers, not strings. This means you cannot simply pass a string into a WebAssembly function.</p><p>To pass a string into WebAssembly, you need to convert the string into numbers (note the TextEncoderAPI specified in webpack.config.js), put the numbers into the memory space of WebAssembly and finally return a pointer of the string to the WebAssembly function so that you can use it in JavaScript. At the end of it, you will need to free up the WebAssembly memory space used by the string.</p><p>If you look at the above JavaScript codes, this is exactly what has been executed automatically. The “helloworld” function first calls “passStringToWasm”. This function creates some memory space in WebAssembly, converts your string to numbers, writes the numbers to the memory space, and returns a pointer to the string. The pointer is then passed to “wasm.helloworld” to perform your JavaScript “alert”. Finally, “wasm.__wbindgen_free” frees up the memory. If you are just passing a simple string, you may be able to do this by yourself, but think of when you have more complex objects and structs, the effort quickly becomes non-trivial. This illustrates the importance of wasm-bindgen in Rust WebAssembly development.</p><p>10. In our previous step, we noted that a “helloworld.js” file is generated by wasm-bindgen, and the functions generated in this file call into our generated WebAssembly codes in “helloworld_bg.wasm”. Basically, “helloworld.js” acts as a bridge between other JavaScripts such as index.js to the generated “helloworld_bg.wasm” WebAssembly.</p><p>We can explore “helloworld_bg.wasm” further by entering the following command:</p><p>wasm2wat helloworld_bg.wasm &gt; helloworld.txt</p><p>This command uses the WebAssemblyBinary Toolkit to translate WebAssembly to WebAssembly text format and saves it to a “helloworld.txt” file. Open “helloworld.txt” in a text editor and look for the “$helloworld” function. This is the generated WebAssembly function of our “helloworld” function that we have defined in “src/lib.rs”.</p><p>Look for the following line in “helloworld.txt”:</p><pre><br>(export &quot;helloworld&quot; (func $helloworld))<br></pre><p>This line exports the “helloworld” wasm function for the host to call. We call this wasm function through “wasm.helloworld” in “helloworld.js”. <br>Next, look for the following line:</p><pre><br>(import &quot;./helloworld.js&quot; &quot;__wbg_alert_7e2b9b91978b2246&quot; (func $__wbg_alert_7e2b9b91978b2246 (type 2)))<br></pre><p>This corresponds to the following JavaScript function generated in “helloworld.js”</p><pre><br>export const __wbg_alert_7e2b9b91978b2246 = function(arg0, arg1)<br></pre><p>This is the part where wasm-bindgen provides a glue to help you use JavaScript functions or the DOM in WebAssembly. With this, we now understand how wasm-bindgen helps us facilitate interaction by importing JavaScript things and exporting WebAssembly things.</p><p>11. Finally, let’s take a look at the other files generated by wasm-bindgen.</p><p>“helloworld.d.ts”</p><p>This .d.ts file contains TypeScript type declarations for the JavaScript glue and is useful if your existing JavaScript application is using TypeScript. You can have your calls into WebAssembly functions type checked or have your IDE provide auto-completion. If you aren’t using TypeScript, you can safely ignore this file.</p><p>“package.json”</p><p>The package.json file contains metadata about the generated JavaScript and WebAssembly package. It automatically fills in all the npm dependencies from your Rust code and enables you to publish to npm.</p><h4>What further understanding can we draw from the above?</h4><p>Take a look at the following code again:</p><p>helloworld.js</p><pre><br>export function helloworld(name) {<br>    const ptr0 = passStringToWasm(name);<br>    const len0 = WASM_VECTOR_LEN;<br>    try {<br>        return wasm.helloworld(ptr0, len0);<br><br>    } finally {<br>        wasm.__wbindgen_free(ptr0, len0 * 1);<br><br>    }<br>}<br></pre><p>The code to allocate and deallocate the memory is carried out for you. There is no need for a Garbage Collector or a full framework engine making WebAssembly applications or modules written in Rust small and optimized. Other languages that require a Garbage Collector will need to include wasm code for their underlying framework engine. So, no matter how optimized they are, their size will not be smaller than what Rust provides. This makes Rust WebAssembly a good choice if you need to integrate or inject small WebAssembly modules into JavaScript web applications.</p><h4>Other things to note beyond the Hello World</h4><p><strong>web-sys</strong></p><p>With wasm-bindgen, we can call a JavaScript function in Rust WebAssembly by using an “extern”. Remember the following from “src/lib.rs”:</p><pre>#[wasm_bindgen] <br>extern &quot;C&quot; { <br>	fn alert(s: &amp;str); <br>}</pre><p>The Web has a huge number of APIs, from DOM manipulation to WebGL to Web Audio. So if our Rust WebAssembly program grows, and we need to make many different calls to Web APIs, we will need to spend time writing a lot of “extern” code. web-sys serves to act as a frontend to wasm-bindgen to provide the raw bindings to all the Web’s APIs. This means if you use web-sys, you can save time by not having to write the “extern” code.</p><p><strong>WASI </strong>(WebAssembly System Interface)</p><p>This is a standardization effort to push WebAssembly beyond the browser. To run WebAssembly on a machine or OS (Operating System) outside the browser, we may need to access system resources. This standardization specifies how you can access these system resources in a secure and yet portable way across different machine architectures. Currently, there are already two implementations, Wasmtime by Mozilla, and Lucet by Fastly. And we need to ask ourselves, will WASM + WASI become the next Docker?</p><p>Brought to you by<a href="https://www.webassemblyman.com/"><strong> WebAssemblyMan.com : Man page of WebAssembly</strong></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a52270009034" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>