<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Manish Sahu on Medium]]></title>
        <description><![CDATA[Stories by Manish Sahu on Medium]]></description>
        <link>https://medium.com/@manishsahu?source=rss-8a884244f467------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*ldlnATKWgvSKgnNW8Dpx3Q.png</url>
            <title>Stories by Manish Sahu on Medium</title>
            <link>https://medium.com/@manishsahu?source=rss-8a884244f467------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Fri, 20 Nov 2026 10:09:55 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@manishsahu/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Working with Geospatial data — Part:4, Cloud Optimized GeoTiffs]]></title>
            <link>https://medium.com/@manishsahu/working-with-geospatial-data-part-4-cloud-optimized-geotiffs-d0bf2f456f19?source=rss-8a884244f467------2</link>
            <guid isPermaLink="false">https://medium.com/p/d0bf2f456f19</guid>
            <category><![CDATA[gis]]></category>
            <category><![CDATA[api]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[geospatial]]></category>
            <category><![CDATA[gdal]]></category>
            <dc:creator><![CDATA[Manish Sahu]]></dc:creator>
            <pubDate>Wed, 08 May 2019 06:02:36 GMT</pubDate>
            <atom:updated>2020-06-26T08:17:57.224Z</atom:updated>
            <content:encoded><![CDATA[<h3>Working with Geospatial data — Part:4, Cloud Optimized GeoTiffs</h3><h3>Introduction</h3><p><strong>What is CoG?</strong> — A cloud optimized GeoTIFF is a regular GeoTIFF file, aimed at being hosted on a HTTP file server. Its internal structure is friendly for HTTP get requests.</p><p>It uses famous Byte serving technique. It is the technology that lets you stream a video or music file online and skip forward or backwards through the content. Instead of having to download the full video file, you can tell the server that you want to start at a particular point.</p><p>The COG format works the same way by allowing users and processes to access just the portion of a raster file that they need.</p><blockquote><strong><em>COG can do the same thing to Geospatial data what youtube did for video streaming</em></strong></blockquote><p><strong>Why CoG?</strong></p><ol><li><strong>Efficient Data Access </strong>— Suppose we have 10GB GeoTIFF stored in cloud server. Now we want to make NDVI map of small portion of dataset.</li></ol><p><strong>Before CoG — </strong>we would have to download whole 10GB dataset then clip your area of interest, then do the processing.</p><p><strong>After CoG —</strong>Download only desired part of dataset and do the processing. It makes process super efficient.</p><p>2. <strong>Reduced Duplication of Data — </strong>Accessing COG’s with cloud workflows enables diverse software to all access a single file online instead of needing to copy and cache the data.</p><p>3. <strong>Image Processing of the fly</strong> — With CoGs, we can create an micro-service in Flask, Django etc to do image processing on the fly. CoG also removes the need of <strong>gdal2tiles</strong> like services to view GeoTIFFs on the browers. Several web based platforms like Planet.com, Digital Globe etc are using CoG to do processing on the fly.</p><p>For Example — Farmshots is using Planet’s CoG dataset to calculate NDVI on the fly. They didn’t donwload any images from planet’s to their servers rather did all the processing on the fly fetching data from Planet’s server.</p><figure><img alt="" src="https://cdn-images-1.medium.com/proxy/0*LkBRRapS6b1IwIF1." /><figcaption>Farmshots applying an NDVI to Cloud Optimized GeoTIFF’s pulled in dynamically from Planet</figcaption></figure><p>4.<strong> Legacy Compatibility — </strong>Traditional GIS software is able to treat Cloud Optimized GeoTIFF’s just like normal GeoTIFF’s, so data providers need only produce one format</p><p><strong>Try it yourself — </strong>Try putting any Cloud Optimized GeoTIFF to a storage service like S3 etc. Just copy and paste the link in the urls given below. You could visualize your dataset on the fly.</p><ol><li><a href="http://www.cogeo.org/map/">http://www.cogeo.org/map/</a></li><li><a href="https://tiles.rdnt.io/">https://tiles.rdnt.io/</a></li></ol><p>Liked it? Do you want to have customization in the processing? Do you want to know how Farmshots like company leverages CoG to do custom processing?</p><p>I have create two Github repository</p><ol><li>CoG converter — This will convert your standard GeoTIFF file to Cloud optimized GeoTIFF. <a href="https://github.com/ManishSahu53/cogconverter">Repo link</a></li><li>Tiling — This repo will help you create an API via which you can do custom processing like visualization of Elevation models, calculate NDVI or even run Deep Neural Networks on the fly. <a href="https://github.com/ManishSahu53/Serving_Rasters_OntheFly">Repo Link</a> [Updated]</li></ol><p>For detailed tutorial on cloud optimized GeoTIFF, refer <a href="https://geoexamples.com/other/2019/02/08/cog-tutorial.html">this</a> link.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*VXi7sPshlEPlKIffENbnww.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*CPE_NIXj133yEidK2Uy1mQ.png" /><figcaption>Visualization of large Elevation model (Here is processing is happening on the fly) and Orthomosaic as a layer on Mapbox</figcaption></figure><p>In case of any doubts, just drop me an email at manishsahu53@live.com. I will try to answer all your queries.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d0bf2f456f19" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Working with GeoSpatial Data — Part:3, Basic data Processing]]></title>
            <link>https://medium.com/@manishsahu/working-with-geospatial-data-part-3-basic-data-processing-53a2a3a91c68?source=rss-8a884244f467------2</link>
            <guid isPermaLink="false">https://medium.com/p/53a2a3a91c68</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[geospatial]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[gdal]]></category>
            <dc:creator><![CDATA[Manish Sahu]]></dc:creator>
            <pubDate>Sat, 06 Apr 2019 12:37:59 GMT</pubDate>
            <atom:updated>2022-10-13T06:59:58.142Z</atom:updated>
            <content:encoded><![CDATA[<h3>Working with GeoSpatial Data — Part:3, Basic data Processing</h3><h3>Introduction</h3><p>In the <a href="https://medium.com/@manishsahu/working-with-geospatial-data-part-1-using-gdal-library-38bdc26ff958">previous Part-1</a> and <a href="https://medium.com/indshine/working-with-geospatial-data-part-2-understanding-file-formats-2b174a5b6354">Part-2</a> blog, we worked with reading and writing of GeoSpatial datasets and understood TIFF file format. I would recommend you to go through it once before reading this.</p><h3>Objectives</h3><p>To introduce basic Geospatial data processing methods such as</p><ol><li>Clipping large dataset into small part using smaller shapefile</li><li>Re-projecting dataset to specified coordinate system</li></ol><h4>Dataset</h4><p>In this tutorial, I will be using a GeoTIFF file and Shapefile downloaded from <a href="https://map.openaerialmap.org/#/30.329614499999973,59.84783641627487,15/latest/5ca86876869304000746b495?_k=ctkxya">OpenAerialMaps</a>. Here is the new <a href="https://oin-hotosm.s3.amazonaws.com/59dc6fc323c8440011d7b29c/0/16346c8c-58c9-446d-8cb7-973fd80e7abd.tif">link.</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*hPpN4dQGnASvuK3v944QSw.png" /><figcaption>GeoSpatial dataset hosted on online</figcaption></figure><h4>Installing Libraries using conda package</h4><pre>conda install gdal<br>pip install numpy<br>pip install geoconvert<br>pip install matplotlib</pre><ol><li><strong>Reprojecting</strong></li></ol><p>Reprojecting here means transforming dataset from one coordinate system to another. You can read more about coordinate system <a href="http://resources.esri.com/help/9.3/ArcGISEngine/dotnet/89b720a5-7339-44b0-8b58-0f5bf2843393.htm">here</a>. Each coordinate system is given an EPSG code. This code contains all the necessary information required to define a coordinate system.</p><p>Let first inspect our dataset using gdalifo</p><pre>gdalinfo City.tif</pre><pre>Driver: GTiff/GeoTIFF<br>Files: City.tif<br>Size is 45276, 24557<br>Coordinate System is:<br>PROJCS[&quot;WGS 84 / UTM zone 21S&quot;]<br>Origin = (385295.154380000021774,6290181.414930000901222)<br>Pixel Size = (0.079250000000000,-0.079250000000000)<br>Metadata:<br>  AREA_OR_POINT=Area<br>Image Structure Metadata:<br>  COMPRESSION=YCbCr JPEG<br>  INTERLEAVE=PIXEL<br>  SOURCE_COLOR_SPACE=YCbCr</pre><p>We can see that our data is in <strong><em>UTM zone 21S</em></strong> coordinate system and upon searching its ESPG code is <strong><em>32721</em></strong>. We wanted to transform its coordinate system to geographic coordinate system having EPSG:4326.</p><p>GDAL library has a method call Warp. It will be used to reproject raster data. Find below a code snippet and complete Jupyter Notebook</p><pre>path_tif = &#39;City.tif&#39;<br>path_dst = &#39;Reprojected.tif&#39;<br>dst_proj = &#39;EPSG:4326&#39;</pre><pre>Destination_datasource = gdal.Warp(path_dst, path_tif, dstSRS=dst_proj, format=&#39;GTiff&#39;)</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/59d2b5954685d6143f1aff4c0c6b86cc/href">https://medium.com/media/59d2b5954685d6143f1aff4c0c6b86cc/href</a></iframe><p>2. <strong>Clipping</strong></p><p>It is often observed that we work on small patch of data instead of whole or we wanted to share only a small of data to a user then Clipping tool would be very handy. We often uses GIS softwares like QGIS/ArcGIS to do such task but this can be very done using python and GDAL library.</p><p>Clipping can have 2 workflows: Exact Clipping, Bounding Box Clipping. These are described in images below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*uJmOfNwjZC_OiA_S6xKNnw.png" /><figcaption>Bounding Box based Clipping vs Exact Boundary Clipping</figcaption></figure><p>Find below a code snippet from jupyter notebook.</p><pre>OutTile = gdal.Warp(path_output, source_datasource, <br>                    format=&#39;GTiff&#39;, <br>                    dstSRS=Projection, <br>                    cutlineDSName=path_shp,<br>                    cropToCutline=True, <br>                    multithread=True)</pre><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/0e690f5dd69c7096cfcc364590ac785c/href">https://medium.com/media/0e690f5dd69c7096cfcc364590ac785c/href</a></iframe><p>All the Notebooks are hosted in Github repo <a href="https://github.com/ManishSahu53/tech_blog/tree/master/notebooks">here</a></p><p>In the next blog, I will be writing about exporting data to multiple file formats. If you have any questions about Geospatial Tech, then you can write me at manishsahu53@live.com. I will be happy to answer all your queries.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=53a2a3a91c68" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What is good for me? Geospatial data formats: GDAL]]></title>
            <link>https://medium.com/@manishsahu/what-is-good-for-me-geospatial-data-formats-gdal-68c3cda4f6aa?source=rss-8a884244f467------2</link>
            <guid isPermaLink="false">https://medium.com/p/68c3cda4f6aa</guid>
            <category><![CDATA[gis]]></category>
            <category><![CDATA[gdal]]></category>
            <category><![CDATA[geospatial]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[cad]]></category>
            <dc:creator><![CDATA[Manish Sahu]]></dc:creator>
            <pubDate>Thu, 04 Apr 2019 11:37:19 GMT</pubDate>
            <atom:updated>2019-04-07T12:58:24.130Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong>Introduction</strong></h3><p>Back in the dim and distant past, the popular 3 formats came about from different proprietary software developers. <strong>GRID</strong> came from <strong>ESRI</strong>. <strong>IMG</strong> came from <strong>ERDAS</strong> and <strong>TIFF</strong> came from <strong>Aldus</strong>. Now there are more than 200 geospatial formats that are used in some form or other. While working in geospatial domain, there are few questions that pop up:</p><ol><li>Which formats to choose</li><li>How to convert to specific format</li><li>Is the format free of cost/open source/ proprietary.</li></ol><h3>Objective</h3><p>Every software (free or paid) uses open-sourced <a href="https://www.gdal.org/">GDAL</a> library extensively. So we will restrict ourself to GDAL supported formats only. The objective of this blog is</p><ol><li>To introduce different types of formats available to us.</li><li>Which format works best for most of the cases.</li></ol><h3>Formats used in various industry</h3><p><strong>GIS Industry</strong></p><p><strong>1.TIFF/TIF/GeoTIFF (Recommended)</strong></p><p>GIS industry often use standard TIF format. TIF can hold internal georeferencing metadata and doesn’t need any external file. Since TIF is so often used, I have made a separate blog on TIF itself <a href="https://medium.com/@manishsahu/working-with-geospatial-data-part-2-understanding-file-formats-2b174a5b6354">here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*le0SaKPtzMRH8eKVX8Y81g.png" /><figcaption>GeoTIFF and KML data hosted on mapbox Platform</figcaption></figure><p><strong>2. VRT</strong></p><p>A virtual raster is a special type of raster layer and referencing file that contains information like reprojection, georeferencing, etc. All the operation written in VRT are applied to TIF. VRT need TIF as supporting file. <strong>For example —</strong> See the figure below. I have performed Merging, warping and hillshade task to a TIF without VRT and with VRT processing pipeline. For detailed analysis refer <a href="http://www.perrygeo.com/lazy-raster-processing-with-gdal-vrts.html">here</a></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/446/1*Pe34htzzjc_WOtfUk78fDQ.png" /><figcaption>Without and with VRT methodology comparison</figcaption></figure><p><strong>3. IMG</strong></p><p>ERDAS Imagine’s IMG files is a proprietary file format developed by Hexagon Geospatial. IMG files are commonly used for raster data to store single and multiple bands of satellite data. For example, this can include file information, ground control points and sensor type.</p><p>Each raster layer as part of an IMG file contains information about its data values. For example, this includes projection, statistics, attributes, pyramids and whether or not it’s a continuous or discrete type of raster.</p><p><strong>4. SHP (Recommended)</strong></p><p>SHP format is creating by ESRI and is open-sourced it. This represents vector data. It has 4 mandatory supporting files : shp, shx, prj, dbf (Attributes data). One special condition for shp file is that you can only save a single type of feature in one shp file. For example- we <strong>cannot</strong> save point and polyline features in the same shp file.</p><p><strong>5. KML</strong></p><p>It is an <a href="https://en.wikipedia.org/wiki/XML">XML</a> notation for geographic annotation and visualization within web based applications. It is extensively used in <a href="https://www.google.com/earth/">Google earth platform</a>. KML can handle multiple feature types (point, line, polygon) in a single file. Only limitation of KML is that it cannot process multiple attributes details effectively. All the attributes are merged</p><p><strong>AutoCAD/CAD Industry</strong></p><p>In CAD industry it is often practically observed that TIFF/IMG/SHP/KML are not at all used. Instead they rely on formats mentioned below. AutoCAD is most widely used CAD software in this industry so I will focus on it only.</p><p><strong>1.PNG (</strong>Not recommended)</p><p>TIFF/Raster is some converted to PNG and then loaded into AutoCAD. PNG will give descent performance for small (&lt;15MB) dataset but fail to work with large dataset. Thus it is not recommended to used PNG format for raster conversion in CAD.</p><p><strong>2. ECW </strong>(Not recommended)</p><p>ECW (<strong>Enhanced Compression Wavelet</strong>) is a proprietary <a href="https://en.wikipedia.org/wiki/Wavelet_compression">wavelet compression</a> image format optimized for aerial and satellite imagery. It was developed by Earth Resource Mapping, and is now owned by <a href="https://en.wikipedia.org/wiki/Intergraph">Intergraph</a> part of <a href="https://en.wikipedia.org/wiki/Hexagon_AB">Hexagon AB</a>. The <a href="https://en.wikipedia.org/wiki/Lossy_compression">lossy compression</a> format efficiently compresses very large images with fine alternating contrast while retaining their visual quality. It overcomes all the limitations of PNG file format. Even large files (&gt;1 GB) can be opened very smoothly on CAD environment.</p><blockquote>The only limitation of ECW is that it is propriety format and user have to <strong>pay to use this format</strong> for reading and writing purposes.</blockquote><p><strong>3. JP2 </strong>(recommended)</p><p>OpenJP2 is an open-source file format that is best of both world of ECW and PNG. Like ECW, CAD can open large JP2 raster files very smoothly and best part is since it is open-source, we can use it for free. It also supports compression and can bring down file size significantly similar to ECW.</p><p>4. <strong>DXF</strong></p><p><strong>AutoCAD DXF</strong> (Drawing Exchange Format) is a CAD data file format developed by <a href="https://en.wikipedia.org/wiki/Autodesk">Autodesk</a> for enabling data interoperability between <a href="https://en.wikipedia.org/wiki/AutoCAD">AutoCAD</a> and other programs. It is vector file format and is supported by various open-source applications.</p><p><strong>Complete list of Supported Formats in GDAL:</strong> With Geo-referencing properties and Maximum file size supported.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/8f99643a56f1d1893edd8da1df34800f/href">https://medium.com/media/8f99643a56f1d1893edd8da1df34800f/href</a></iframe><p>I would love to hear the kind of format you are using in your organisation. Write me at manishsahu53@live.com so that we can discuss on some geospatial tech.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=68c3cda4f6aa" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Working with GeoSpatial Data — Part:2, Understanding file formats]]></title>
            <link>https://medium.com/@manishsahu/working-with-geospatial-data-part-2-understanding-file-formats-2b174a5b6354?source=rss-8a884244f467------2</link>
            <guid isPermaLink="false">https://medium.com/p/2b174a5b6354</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[gis]]></category>
            <category><![CDATA[tiff]]></category>
            <category><![CDATA[geospatial]]></category>
            <category><![CDATA[gdal]]></category>
            <dc:creator><![CDATA[Manish Sahu]]></dc:creator>
            <pubDate>Sat, 30 Mar 2019 14:56:39 GMT</pubDate>
            <atom:updated>2019-04-06T10:51:02.633Z</atom:updated>
            <content:encoded><![CDATA[<h3>Working with GeoSpatial Data — Part:2, Understanding file formats</h3><h3>Introduction</h3><p>In the <a href="https://medium.com/@manishsahu/working-with-geospatial-data-part-1-using-gdal-library-38bdc26ff958">previous Part-1</a> blog, we worked with reading and writing of GeoSpatial datasets. There are multiple file formats that support geo-referencing. In Raster formats like TIF, IMG, VRT, ECW, JP2 etc can support some kind of georeferencing. The most famous and widely used is the TIF format. It supports internal georeferencing and not extra supporting file (tfw, pgw, prj etc) is needed to work with. All the other formats need supporting files.</p><blockquote><strong><em>Note1</em></strong> — TIF is known in multiple names like TIFF, GeoTIFF, GeoTIF. But these all mean the same.</blockquote><h3>Attributes</h3><ol><li><strong>Supporting data types</strong></li></ol><p>TIF also supports multiple data types like Byte, UInt16, Int16, UInt32, Int32, Float32, Float64, CInt16, CInt32, CFloat32 and CFloat64 are supported for both reading and writing.</p><p>2. <strong>Transparency</strong></p><p>TIF files can contain internal transparency masks. It means instead of showing black pixels where there is no data, transparency of image is maintained. See image below to understand this better.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0tk6KSMpSiWijbEWpAQqTQ.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GbCn7w9rWncZp3ogiLrpAg.png" /><figcaption><strong>Image 1 </strong>— It doesn’t account for transparency and black pixels are displayed where there is nodata<strong>, Image 2 </strong>— It properly account for transparency and no pixels are displayed.</figcaption></figure><p>3. <strong>Overviews</strong></p><p>Overviews or Pyramids are low-resolution level of detail data which is used to improve the performance of software opening TIF. ESRI <a href="https://desktop.arcgis.com/en/arcmap/10.3/manage-data/raster-and-images/raster-pyramids.htm">here</a> explained this very well. The GeoTIFF driver supports reading, creation, and update of internal (along with TIF, no extra file) overviews.</p><p>4. <strong>Metadata</strong></p><p>This is the most important attribute of TIF dataset. It contains georeferencing information, Bounds, Size, Projection system, Pixel Size, etc. See below to get the details of the metadata object. For example — PROJCS represents a coordinate system of TIF.</p><pre><strong>Driver</strong>: GTiff/GeoTIFF<br><strong>Files</strong>: Ortho.tif<br><strong>Size</strong> is 62223, 45930<br><strong>Coordinate</strong> <strong>System</strong> is:<br><strong>PROJCS</strong>[&quot;WGS 84 / UTM zone 44N&quot;,<br>    GEOGCS[&quot;WGS 84&quot;,<br>        DATUM[&quot;WGS_1984&quot;,<br>            SPHEROID[&quot;WGS 84&quot;,6378137,298.257223563,<br>                AUTHORITY[&quot;EPSG&quot;,&quot;7030&quot;]],<br>            TOWGS84[0,0,0,0,0,0,0],<br>            AUTHORITY[&quot;EPSG&quot;,&quot;6326&quot;]],<br>        PRIMEM[&quot;Greenwich&quot;,0],<br>        UNIT[&quot;degree&quot;,0.0174532925199433],<br>        AUTHORITY[&quot;EPSG&quot;,&quot;4326&quot;]],<br>    PROJECTION[&quot;Transverse_Mercator&quot;],<br>    PARAMETER[&quot;latitude_of_origin&quot;,0],<br>    PARAMETER[&quot;central_meridian&quot;,81],<br>    PARAMETER[&quot;scale_factor&quot;,0.9996],<br>    PARAMETER[&quot;false_easting&quot;,500000],<br>    PARAMETER[&quot;false_northing&quot;,0],<br>    UNIT[&quot;metre&quot;,1,<br>        AUTHORITY[&quot;EPSG&quot;,&quot;9001&quot;]],<br>    AUTHORITY[&quot;EPSG&quot;,&quot;32644&quot;]]<br><strong>Origin</strong> = (391824.812343188037630,2563954.159405739512295)<br><strong>Pixel</strong> <strong>Size</strong> = (0.049987000000000,-0.049987000000004)<br><strong>Metadata</strong>:<br>  AREA_OR_POINT=Area<br><strong>Image</strong> <strong>Structure</strong> <strong>Metadata</strong>:<br>  <strong>COMPRESSION</strong>=LZW<br>  <strong>INTERLEAVE</strong>=PIXEL<br>Corner Coordinates: ...<br><strong>Band</strong> <strong>1</strong> Block=256x256 Type=Byte, ColorInterp=Red<br>  <strong>Overviews</strong>: 31112x22965, 15556x11483, 7778x5742, 3889x2871, 1945x1436, 973x718, 487x359, 244x180</pre><p>5. <strong>Compression</strong></p><p>TIF supports numerous compression JPEG, LZW, etc. A complete list can be found <a href="https://www.gdal.org/frmt_gtiff.html">here</a>. Compression can be lossy like JPEG or lossless like LZW. Lossy compression is best suited for RGB colored 3-band TIFs. Compression can be very useful and brings down size of file drastically. For example — A compressed RGB TIF size is 169MB and uncompressed size is 3.40 GB. We can get compression method from metadata of TIF file as given above.</p><blockquote><strong>Note2 </strong>— Doing JPEG will remove transparency from the TIF</blockquote><p>6. <strong>BigTiFF</strong></p><p>Maximum size a normal TIF file can support is 4GB. Above it, BIGTiFF is used. Although it acts almost similar to normal TIF and we shouldn’t worry much about it. Just remember to check BIGTIFF yes when saving a large TIF file.</p><p>7. <strong>Streaming</strong> <strong>operations</strong></p><p>This is the most interesting part of TIF, we can actually read TIF dataset directly from the web without downloading of local PC. I will cover this topic in detail in the late blogs.</p><p>Now, that you have a good understanding of TIF dataset. We can <strong>read</strong> and <strong>manipulate</strong> these attributes using <strong>GDAL </strong>python library too. In the next blog, I will cover how to read these attributes and also manipulate some of it using python. You can write to me at manishsahu53@live.com if you have any questions regarding Geospatial tech.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2b174a5b6354" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Working with GeoSpatial Data — Part:1 using GDAL library]]></title>
            <link>https://medium.com/@manishsahu/working-with-geospatial-data-part-1-using-gdal-library-38bdc26ff958?source=rss-8a884244f467------2</link>
            <guid isPermaLink="false">https://medium.com/p/38bdc26ff958</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[gdal]]></category>
            <category><![CDATA[geospatial-data]]></category>
            <category><![CDATA[geospatial]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Manish Sahu]]></dc:creator>
            <pubDate>Wed, 27 Mar 2019 09:49:16 GMT</pubDate>
            <atom:updated>2019-04-06T10:47:41.033Z</atom:updated>
            <content:encoded><![CDATA[<h3>Working with GeoSpatial Data — Part:1 using GDAL library</h3><h3>Introduction</h3><p>As with most data, being able to analyse, query, transform, and visualize <a href="https://developers.planet.com/planetschool/geospatial-data/">geospatial data</a> makes your work easier and more useful. If you’re new to working with and using geospatial data, though, the world of geo-tools available can be overwhelming.</p><p>In this series, we’ll learn about a powerful, widely-used, and broadly applicable Free &amp; Open Source Software (FOSS) tool in python that will empower you to do more with your data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Rt4-ggPi1moEJnrux9N11Q.png" /><figcaption>GeoSpatial data hosted on www.indshine.com</figcaption></figure><h3><strong>Objectives</strong></h3><p>The objective of this series is to :</p><ol><li>Introduce to most popular geospatial libraries like GDAL</li><li>Give hands on experience working with GeoSpatial data.</li></ol><h3><strong>Getting Started</strong></h3><p>Geospatial data can be of two forms<a href="https://gisgeography.com/spatial-data-types-vector-raster/"> <strong>Vector </strong>and<strong> Raster</strong></a><strong>. GDAL </strong>can work with both vectors and raster formats. Although several wrappers like rasterio, fiona also exist that deal specifically with the formats.</p><p><strong>Install GDAL</strong></p><ol><li>Download Miniconda distribution for python3 from <a href="https://docs.conda.io/en/latest/miniconda.html">here</a>.</li><li>Install GDAL using</li></ol><pre>conda install gdal</pre><p><strong>Working with Rasters</strong></p><ol><li>Reading TIF data</li></ol><p>Georeferenced TIF data can confusing to mange and process. Parsing it in numpy array will give us flexibility to manipulate it. Example below will parse TIF data to array.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/286e4942f46a6b01f536bcd9fef68e8b/href">https://medium.com/media/286e4942f46a6b01f536bcd9fef68e8b/href</a></iframe><p>2. After reading it as numpy array, and then processing it as per your needs, We need to write this processed data back to georeferenced TIF. Writing numpy array as georeferenced TIF follow code below.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/5960e3ee1966f1fbd393b56917a7827b/href">https://medium.com/media/5960e3ee1966f1fbd393b56917a7827b/href</a></iframe><pre><strong>Note -1</strong> You must assign outdata = None after flushing data to disk. Else you will face problem of corrupted data saving.</pre><pre><strong>Note -2 </strong>While writing we often take a geo-referenced dataset, do some processing then again it to same referenced coordinate system.</pre><p>3. Reading/Writing TIFs <strong>larger</strong> <strong>than</strong> <strong>Memory</strong> (RAM)</p><p>There is often cases when TIFs size is so huge that it can’t be loaded in the memory at once. So to do processing of such datasets, you can refer to snippet below. This kind of approach gives us freedom to work on TBs of data.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/f4d7ac435ae81cc1f7bd05fba33a5c68/href">https://medium.com/media/f4d7ac435ae81cc1f7bd05fba33a5c68/href</a></iframe><p><strong>Working with Vectors</strong></p><ol><li>Reading vector data (SHP)</li></ol><p>Here OGR is used which comes install with GDAL library by default.</p><p>Features- list containing geometry (coordinate information)</p><p>Attributes- list containing dict of attributes</p><p>SpatialRef- Coordinate system reference</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/af67cdcde44f142bfcfa201c72b36c51/href">https://medium.com/media/af67cdcde44f142bfcfa201c72b36c51/href</a></iframe><p><strong>Bonus</strong></p><ol><li>Merging/Mosaic muliple rasters:</li></ol><p>It is often observed that we save large data into number of small manageable files. It make dataset easy to share/upload. But If you want to put these pieces together then, mosaicing/merging is required. You can achieve it easily and steps are given below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/535/0*wrJAY1CmnCmduUiz.png" /><figcaption>Mosaicing/Merging dataset</figcaption></figure><p>To merge multiple rasters, we need to simply combine all the rasters.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/c4b1158f01d491b2d2288afe861dad8c/href">https://medium.com/media/c4b1158f01d491b2d2288afe861dad8c/href</a></iframe><pre><strong>Note -3</strong> list_tif = list of path of individual tifs</pre><p>Let’s connect over email, write me at mainshsahu53@live.com, if you have any questions regarding Geospatial tech. We can also discuss on either of our projects.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=38bdc26ff958" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Regulating UAV industry in India]]></title>
            <link>https://medium.com/@manishsahu/regulating-uav-industry-in-india-965d4e87dbf5?source=rss-8a884244f467------2</link>
            <guid isPermaLink="false">https://medium.com/p/965d4e87dbf5</guid>
            <category><![CDATA[uav-industry]]></category>
            <category><![CDATA[india]]></category>
            <category><![CDATA[drones]]></category>
            <category><![CDATA[drone-tips]]></category>
            <category><![CDATA[regulation]]></category>
            <dc:creator><![CDATA[Manish Sahu]]></dc:creator>
            <pubDate>Tue, 28 Aug 2018 06:00:23 GMT</pubDate>
            <atom:updated>2019-01-01T11:11:04.697Z</atom:updated>
            <content:encoded><![CDATA[<p>India has finally permitted commercial flying of drones with effect from December 1, 2018, according to a <a href="http://pib.nic.in/newsite/PrintRelease.aspx?relid=183093">press</a> release from the Ministry of Civil Aviation on August 27 2018. This has come after <a href="http://dgca.nic.in/public_notice/PN_UAS.pdf">a Public Notice</a> back in October 2014 then with <a href="http://www.dgca.nic.in/misc/draft%20circular/AT_Circular%20-%20Civil_UAS(Draft%20April%202016).pdf">First </a>and <a href="http://www.dgca.nic.in/misc/draft%20cars/CAR%20-%20UAS%20(Draft_Nov2017).pdf">Second </a>DGCA Drone Regulation Draft in April 2016 and November 2017.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zL-iAt9wtdZ5s5ay0n-R7w.jpeg" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*RLwxAPxpnLuNKANtljmaYw.jpeg" /><figcaption>Suresh Prabhu, Drone</figcaption></figure><h3><strong>Terminologies</strong></h3><ul><li><strong>UAV </strong>— Unmanned Aerial Vehicle</li><li><strong>UIN </strong>— Unique Identification Number. UIN is a unique number given to every UAV. This is similar to number plate for automobiles.</li><li><strong>ATC </strong>— Air Traffic Control. It will keep the record, monitor and control air space.</li><li><strong>AGL</strong>— Above Ground Level.</li><li><strong>NPNT </strong>— No Permission, No Takeoff. This is a software which denies UAV to takeoff if permission of that particular flight is not granted.</li><li><strong>PPL </strong>— Private Pilot License. It is type of pilot licence that allows the individual to act as pilot in command of an UAV privately</li><li><strong>RPAS </strong>— Remotely Piloted Aircraft System(s). This term refers to the UAV/Drones that can be controlled using Remote controller.</li><li><strong>VLOS </strong>— Visual Line Of Sight. If UAV can be seen with a naked eye, then it is in VLOS.</li><li><strong>Airspaces </strong>— <strong>Red</strong> (flying not permitted), <strong>yellow </strong>(controlled airspace) and <strong>green </strong>(automatic permission).</li><li><strong>UAOP </strong>— Unmanned Aircraft Operator Permit is required to for takeoff.</li><li><strong>NTRO </strong>— National Technical Research Organisation.</li><li><strong>DSP </strong>— Digital Sky Platform. This is an online portal where registrations, permissions, incident reporting etc will be carried out.</li></ul><h3><strong>Categories </strong>of UAVs:</h3><ul><li>Nano — up to 250 grams</li><li>Micro — 250 grams to 2 kgs</li><li>Small — 2 kgs to 25 kgs</li><li>Medium — 25 kgs to 150 kgs</li><li>Large — above 150 kgs</li></ul><h3><strong>Feature points</strong></h3><ol><li>All RPAS except nano and those owned by NTRO, ARC and Central Intelligence Agencies are to be registered and issued with UIN.</li><li>An UAOP permit will be required to fly the UAV except for flying nano drone below 50ft (15.24m) and micro drone for flying at less than 200ft (60.96m) and those owned by NTRO, ARC, Central Intelligence Agencies. UAOP will be valid for Five years</li><li>Mandatory equipment required for operation of drones except for nano drones are</li></ol><ul><li>GNSS(GPS)</li><li>Return to Home (RTH),</li><li>Flight controller with flight data logging capabilities,</li><li>RF-ID and SIM Card</li><li>NPNT functionality</li><li>Fire resistant identification plate inscribed with UIN</li><li>SSR transponder (Mode ‘C’ or ‘S’) or ADS-B OUT equipment</li><li>Barometric equipment with capability for remote sub-scale setting</li><li>Geo-fencing capability</li><li>Detect and Avoid capability.</li></ul><p>4. UAVs are to be operated within VLOS during daytime only and upto 400ft</p><p>5. Flying in controlled(yellow) airspace, filling of flight plan and obtaining air defence clearance (ADC)/ Flight information centre(FIC) number shall be necessary.</p><p>6. Minimum manufacturing standards and training requirements of Remote Pilot of small and above categories.</p><p>7. Foreigners are currently not allowed to fly drones in India. For commercial purpose, they need to lease RPAS to an Indian entity who in-turn will obtain UIN and UAOP from DGCA.</p><p>8. The operator of all RPA except Nano RPA will notify any incident/accident involving RPA to the Director of Air Safety, DGCA through DSP.</p><p>9. Pilot license is not required to for flying UAV as of now but you need to undergo minimum training requirements specified in the Civil Aviation Requirements on “Requirements for Operation of Civil RPAS”</p><p>10. Pilot must be atleast 18 years old.</p><p>11. All individuals / operators (except Nano), shall maintain records of each RPA flight records and should be produced on demand by DGCA.</p><p>12. RPAS can be used for agricultural purpose except for the purpose of spraying pesticides until specifically cleared.</p><p>13. DGCA has not stipulated any requirement on minimum amount of insurance for any category of drone. It should be of an adequate amount to cover the risks, damages or other factors that is posed by operation of drone. Further, it is an agreement between the insurance company and the policy holder.</p><p>14. Violation of compliance to any of the requirements and falsification of records/documents shall attract penal action including imposition of penalties as per applicable IPCs (such as 287, 336, 337, 338 or any relevant section of IPC) or Aircraft Act 1934 or Aircraft Rules 1937 or any statutory provisions.</p><p>15. Any kind of delivery (food, packages etc), human transportation are not allowed.</p><p>For complete list of FAQs, check this <a href="http://dgca.nic.in/cars/RPS-FAQs.pdf">link</a>.</p><h3><strong>Unanswered Questions</strong></h3><ol><li>Who will provide training and certifications and how to obtain it?</li><li>What are minimum manufacturing standards for UAVs?</li><li>What if flight is to be taken in remote areas where internet signals are not available?</li><li>Above ground level 400ft limit is with respect to Home location or is it according to terrain?</li></ol><p>DGCA will issue the Drone Regulation 2.0 which would include certification of Drone Hardware and Software, Automated Airspace Management, Beyond Visual Line of Sight (BVLOS) Operations, India’s contribution to global standards and any amendments needed.</p><p>“Today we start an exciting new chapter in India’s aviation history by allowing commercial use of drones. I am sure that many new and exciting applications will emerge that will propel India’s economy forward. Our progressive regulations will encourage a vast Made in India drone industry,” Minister of Civil Aviation, Shri Suresh Prabhu said.</p><h3>About Indshine</h3><p><a href="http://www.indshine.com/"><strong>Indshine</strong></a> provides geospatial insights and design solutions for Highways, Railways, Mining, Forestry, and Power Transmission. We also create digital maps and 3D models by capturing data through drones, satellites and LiDARs. These are widely used in smart city planning, surveying of a given site, monitoring the site progress, and inspecting various kinds of structures such as bridges, dams, pipelines, high walls, etc.</p><p>Moreover, based on the industry’s requirements, we have developed a project collaboration platform called<strong> </strong><a href="http://felix.indshine.com/"><strong>Felix (felix.indshine.com)</strong></a><strong> </strong>which allows all the stakeholders to visualise the area of interest, generate spatial insights, discuss with other members, and make informed decisions quickly.</p><p>For more information, kindly send us an email at <a href="/service/mailto:info@indshine.com"><strong>info@indshine.com</strong></a><strong> </strong>or visit <a href="http://www.indshine.com/"><strong>www.indshine.com</strong></a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=965d4e87dbf5" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[What are the outputs obtained in aerial surveying using drones?]]></title>
            <link>https://medium.com/@manishsahu/what-are-the-outputs-obtained-in-aerial-surveying-using-drones-22e576cb3df8?source=rss-8a884244f467------2</link>
            <guid isPermaLink="false">https://medium.com/p/22e576cb3df8</guid>
            <category><![CDATA[3d-mesh]]></category>
            <category><![CDATA[mapping]]></category>
            <category><![CDATA[orthomosaics]]></category>
            <category><![CDATA[digital-elevation-model]]></category>
            <category><![CDATA[point-cloud]]></category>
            <dc:creator><![CDATA[Manish Sahu]]></dc:creator>
            <pubDate>Mon, 28 May 2018 06:37:59 GMT</pubDate>
            <atom:updated>2019-01-01T11:12:41.298Z</atom:updated>
            <content:encoded><![CDATA[<h4>Drones (also called as UAVs)are getting popular in engineering applications, but what do we get out of it?</h4><p>In aerial surveying, we find that the practitioners use a lot of technical words which seem to be confusing most of the times. Let us look at some of the them which are the most popular ones:</p><h3><strong><em>1. Orthomosaic</em></strong></h3><p>Orthomosaic is made from two words Ortho (meaning perpendicular) and Mosaic (meaning small pieces joined together to make a big piece). An <strong>orthophoto,</strong> <strong>or orthomosaic</strong> is a large image of an area which can be used to measure actual distances between any two given points on the image. A normal photograph, as taken from above is not uniform in terms of scale (It means that the ratio of the distance between two points on a photo to the actual distance between the same two points on the ground is not equal). However, an orthomosaic is such that the scale of the image to the actual ground is uniform, thus representing the Earth’s actual surface.</p><blockquote>In simple words, an orthomosaic is an accurate representation of the Earth’s surface from top view in 2D.</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*1zZeJLOmHf4JV2gIeiz9gg.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WNcIEgz4uTgEUV5ewDnD6A.png" /><figcaption>Orthomosaic of a railway line under construction superimposed on Google Earth. (Image on the right is an enlarged version of the left one.)</figcaption></figure><h3>2. <strong><em>Digital Elevation Model</em></strong></h3><p><em>Elevation simply means height. In simplest terms, it can be referred to as Digital Height Model. </em>Thus, a digital elevation model (<strong>DEM</strong>) is a representation of a <a href="https://en.wikipedia.org/wiki/Terrain">terrain</a>’s surface only in terms of its height. DEMs are the superset of digital surface models (DSM) and digital terrain models (DTM).</p><p><strong>Digital Surface Model (DSM): </strong>As the name suggests, a digital surface model is a digital representation of the Earth’s surface (in terms of height). In other words, DSM provides the height of all the points on Earth as seen from above such as buildings, trees etc. It can be obtained directly from any standard photogrammetric software.</p><p><strong>Digital Terrain Model (DTM): </strong>It is similar to a digital surface model except it represents only the bare surface of the earth without including any height information about the structures above ground such as buildings, trees. It is a bare earth model. A DTM can be obtained from further processing of a DSM, and is used to generate <a href="https://en.wikipedia.org/wiki/Contour_line">contour</a> of the given area.</p><h3>3. <strong><em>Point Cloud</em></strong></h3><p>Imagine the real-world in form of points. That’s what a point cloud is — a 3D representation of the real-world in form of points. In other words, it is a collection of large number of points/pixels (&gt;1 crore) where each point has X, Y and Z location in a given 3D coordinate system, and also a corresponding R, G, B colour value, such that all the points when combined according to their position in the coordinate system, it looks a digital copy of the real-world. Unlike Orthomosaic, where we can do only 2D measurement (distance/length), Point Cloud gives us the freedom of measuring in 3-dimension (3D), and hence enabling us to find out height difference between two points, volume estimation of a specified region, elevation profile of a section etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dm6f55h4gBAITnBNhCiC-Q.png" /><figcaption>Point Cloud showing Railway Station under construction</figcaption></figure><h3>4. <strong><em>3D Mesh</em></strong></h3><p>A 3D Mesh is the most accurate representation of the real-world in every form. Unlike point cloud, which is a discrete set of points, a 3D Mesh is continuous similar to the real-world. It also gives us the freedom of measuring in 3-dimension (3D), and hence enabling us to find out height difference between two points, volume estimation of a specified region, elevation profile of a section etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*5Fu-QLV0PTymiLtuf4Q4cA.png" /><figcaption>3D Mesh of Kashi Railway Station</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*a-n4XOshSIk9cfMGvU9hHA.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*vVRRcgX5t7a3APHYIzurGw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GtjpAaq4VSmBSGFamdf7jQ.png" /><figcaption>3D Mesh of Jaipur (Wireframe View showing the mesh of triangles which creates the 3D structure)</figcaption></figure><h3>About Indshine</h3><p>Indshine provides enterprise drone solutions to highways, railways, mining, smart city, power transmission, forestry etc. Please visit <a href="http://www.indshine.com/">www.indshine.com</a> to know more. If you have any query or want a demo, kindly drop us an email at info@indshine.com.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=22e576cb3df8" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>