<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Nirmal Sarswat on Medium]]></title>
        <description><![CDATA[Stories by Nirmal Sarswat on Medium]]></description>
        <link>https://medium.com/@nirmalsarswat?source=rss-9ac740e18768------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/0*iyMShlYByWJgnGrh.</url>
            <title>Stories by Nirmal Sarswat on Medium</title>
            <link>https://medium.com/@nirmalsarswat?source=rss-9ac740e18768------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 15 Nov 2026 22:35:15 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@nirmalsarswat/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Summer of Code 2018 — Week 8 — File transfer server in Flask, Python]]></title>
            <link>https://medium.com/@nirmalsarswat/summer-of-code-2018-week-8-file-transfer-server-in-flask-python-d1eb2d3347ba?source=rss-9ac740e18768------2</link>
            <guid isPermaLink="false">https://medium.com/p/d1eb2d3347ba</guid>
            <category><![CDATA[flask]]></category>
            <category><![CDATA[google-summer-of-code]]></category>
            <category><![CDATA[python]]></category>
            <dc:creator><![CDATA[Nirmal Sarswat]]></dc:creator>
            <pubDate>Sun, 24 Jun 2018 13:34:15 GMT</pubDate>
            <atom:updated>2018-06-24T13:34:15.025Z</atom:updated>
            <content:encoded><![CDATA[<p>On this data world, files, databases are very crucial things to manage. <a href="http://www.scorelab.org/">scorelab</a> is developing a tool — <a href="https://github.com/scorelab/bassa"><strong>Bassa</strong></a><strong>, which will manage files in a smart way and utilize internet bandwidth in the best way.</strong><br>From last two week I was trying to implement a mechanism to transfer the files from server to browser client where backend is written in Python using Flask micro-framework. Here is what I used to make something robust and enough scalable.</p><p>Architecture : <br>1. File transfer invoicing API<br>2. Compression of file/files which would be sent over internet<br>3. Acknowledgement API for compression status<br>4. Document transfer protocol from browser client to server<br>5. Sending compressed file in response :)</p><p><strong>File Transfer Invoicing API : <br></strong>Whenever user want to download the file, first we should initiate compression of files and then transfer compressed file to consume less internet. On file transfer invoicing API hit, we will get some file related details, compress it and store to a temporary folder which later we will delete in a smart way.</p><p>Here is some code for each and every step : <br> <strong>File Invoicing API Endpoint:</strong></p><pre>@server.route(&#39;/api/download&#39;, methods=[&#39;POST&#39;])<br>def add_download_request():<br>    token = token_validator(request.headers[&#39;token&#39;])<br>    if token is not None:<br>        # some logical way to decide a unique id for a request,    something like UUID based<br>        # and we will run a subprocess to compress the file<br>        subprocess.call(&#39;python compression.py &lt;all required arguments goes here&gt; &amp;&#39;) # &amp; for daemon process<br>        return generate_unique_id(&#39;&lt;some file related key&gt;&#39;)</pre><pre>    elif token is not None:<br>        return &#39;{&quot;error&quot;:&quot;not authorized&quot;}&#39;, 403<br>    else:<br>        return &#39;{&quot;error&quot;:&quot;token error&quot;}&#39;, 403</pre><p><strong>Compression of files : <br></strong> We would be using zlib and zipfile libraries in Python for compression work. <br>A simple script, <em>compression.py</em><strong><em>, </em></strong>for compression work where arguments like file_list and file_name would be passed from subprocess call on main thread</p><pre>def compress(file_list, file_name):<br>   try:<br>      zip_path = get_zip_path(file_name)<br>      zip_file = zipfile.ZipFile(zip_path, &#39;w&#39;, zipfile.ZIP_DEFLATED)<br>      for files in file_list:<br>         zip_file.write(files, basename(files))<br>      zip_file.close()<br>      logging.info(&#39;Compression of files is done, written to following file {0}&#39;.format(zip_path))<br>      return zip_path<br>   except Exception as e:<br>      logging.error(&#39;Exception occurred during compression {0}&#39;.format(e)) # some logging stuffs</pre><pre>def dl_config_reader():<br>   script_path = os.path.abspath(os.path.dirname(__file__))<br>   path = os.path.join(script_path, &quot;../dl.conf&quot;) # path configs<br>   conf_str = &#39;&#39;<br>   with open(path, &#39;r&#39;) as conf_file:<br>      conf_str = conf_file.read()<br>      conf_file.close()<br>   return json.loads(conf_str)<br><br><br>def get_zip_path(file_name):<br>   conf_json = dl_config_reader()<br>   tmp_dir = conf_json[&#39;tmp_folder&#39;]<br>   if not os.path.exists(tmp_dir):  # create temp directory for zip<br>      os.makedirs(tmp_dir)<br>   zip_path = tmp_dir + &#39;/&#39; + file_name + &#39;.zip&#39;<br>   return zip_path</pre><p><strong>Acknowledgement API for compression status : <br></strong>A database mapping could be useful in this case where you will change the status of file compression from it’s unique id. A browser client will ask for file as soon server responds compression is done for acknowledgement API.</p><p><strong>Document transfer protocol from browser client to server and send the file in response : <br></strong>For this we would be using simple Flask in-built <a href="http://flask.pocoo.org/docs/0.12/api/#flask.send_file"><strong>send_file</strong></a><strong> </strong>which binds file in IO streams and browser acknowledge as document/file with it’s tags.<br>A simple code for this : (All the functions name are according to their work)</p><pre>def send_file_from_path():<br>   token = token_validator(request.args[&#39;token&#39;])<br>   if token is not None:<br>      try:<br>         gid = request.args.get(&#39;gid&#39;)<br>         logging.info(&#39;User asked for file, having GID &#39; + gid)<br>         id = get_id_from_gid(gid)<br>         download_path = get_download_path(id)<br>         download_name = get_download_name_from_id(id)<br>         file_path = compression_handler.compress(file_list=[download_path], file_name=download_name)<br>      except Exception as e:<br>         logging.error(&quot; get file API (/api/get/file) got wrong arguments, thrown an error :: %s&quot; % e)<br>         return Response(&quot;error&quot;, status=200)<br>      <br>      try:<br>         download_name += &#39;.zip&#39;<br>         return send_file(filename_or_fp=file_path, attachment_filename=download_name<br>                      , as_attachment=True)<br>      except Exception as e:<br>         logging.error(&quot;File sending has a exception. When sending file, we got :: %s&quot; % e)<br>         return Response(&quot;File you are trying to access is not available to us. Please ask admin to check the server&quot;<br>                     , status=200)<br>   else:<br>      return Response(&quot;Invalid Token in request&quot;, 403)</pre><p><a href="http://flask.pocoo.org/docs/0.12/api/#flask.send_file"><strong>send_file</strong></a><strong> </strong>will need the path of file, the attachment type and file name.</p><p>More robust way would be implementing WSGI techniques for request handling and invoking zlib with 9th level compression in short timespan compression. Zipfile default use zlib for compression things but it can’t compress to the same which zlib can do for I/O streams.</p><p>Thank you for your time, ping me, comment me for any help :)</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d1eb2d3347ba" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>