<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Sankemax (Maxim Golman) on Medium]]></title>
        <description><![CDATA[Stories by Sankemax (Maxim Golman) on Medium]]></description>
        <link>https://medium.com/@sankeman2004?source=rss-6e6d97e59831------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*QOc87sLgqJqhe6HROm6vKw.jpeg</url>
            <title>Stories by Sankemax (Maxim Golman) on Medium</title>
            <link>https://medium.com/@sankeman2004?source=rss-6e6d97e59831------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Tue, 17 Nov 2026 13:28:09 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@sankeman2004/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Working with schemas in a schemaless-ish world]]></title>
            <link>https://levelup.gitconnected.com/working-with-schemas-in-a-schemaless-ish-world-d8e878d2ed94?source=rss-6e6d97e59831------2</link>
            <guid isPermaLink="false">https://medium.com/p/d8e878d2ed94</guid>
            <category><![CDATA[npm]]></category>
            <category><![CDATA[schema]]></category>
            <category><![CDATA[json]]></category>
            <category><![CDATA[mapping-tool]]></category>
            <category><![CDATA[js]]></category>
            <dc:creator><![CDATA[Sankemax (Maxim Golman)]]></dc:creator>
            <pubDate>Tue, 21 Apr 2020 18:43:45 GMT</pubDate>
            <atom:updated>2020-04-22T13:28:48.526Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*03dce1GaHPOVqfLntwAWmw.jpeg" /></figure><p><strong><em>TL;DR</em></strong><em>: I’ve written a JS library that makes life easier when converting between different JSON structures and working with undocumented or unreliable schemas. Check it out on my</em><strong><em> </em></strong><em>companies’</em><strong><em> </em></strong><a href="https://github.com/intelligogroup/object-to-schema-mapper"><strong><em>github</em></strong></a><strong><em>.</em></strong></p><p>Web-developers face the need of working with external APIs at a regular basis. Sometimes those APIs are poorly documented while others respond with a changing structure based on different request input.</p><p>When we — back-end developers — create client applications that serve our internal needs, we have the full control over the schema we work with, but when we go out to the open, we encounter all kinds of wild animals.</p><p>I had the pleasure of collecting data from different API providers only to process and convert them all into a single in-house schema. Sometimes, even adding additional APIs as sources to an already active schema that is being used in production code.</p><ul><li>How is it possible to work with definitive schemas on your end, while <strong>not </strong>worrying that 3rd party APIs don’t even bother documenting their schemas?</li><li>How can I free myself from hard-coded and complicated conversions between one schema and another?</li><li>Can I quickly test my assumptions about the 3rd-party API schema? Does the current conversion from the API response to our internal schema is missing something?</li></ul><p>These aren’t theoretical questions. It was a serious concern at Intelligo-group - the company I work in - and I have written a JS library that tries to cope with the problems mentioned above.</p><p><strong>The library is divided into three main parts:</strong></p><ol><li>Automatic schema creation out of a 3rd party response.</li><li>Mapping — from one JSON structure to another — is based on easy-to-write and human readable configuration.</li><li>Testing — create an exemplar data from schema creation (1) and apply the mapping (2) to check whether the result is satisfactory.</li></ol><p><strong>The following part is an example how to use the library:</strong></p><p>For the sake of the example, let’s assume I’m a movie enthusiast and I want to collect all needed information about a movie. For that matter, I use an imaginary API of an imaginary site and this is the response I get for searching “Avengers: Infinity War”:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/47a09f77c939506a4d73b879c5590f21/href">https://medium.com/media/47a09f77c939506a4d73b879c5590f21/href</a></iframe><p>I would like to use my library and create a schema from the response:</p><ol><li>Let’s install the library: npm i @intelligo.ai/object-to-schema</li></ol><p>2. And invoke the schema creator:</p><pre>import response from &#39;./response.json&#39;;<br>import { mapObjectToSchema } from &#39;@intelligo.ai/object-to-schema&#39;;</pre><pre>const schema = mapObjectToSchema(response);<br>console.log(JSON.stringify(schema, null, 2));</pre><p>The output we’ll get is:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b777863291272fa629537a423bc3803b/href">https://medium.com/media/b777863291272fa629537a423bc3803b/href</a></iframe><p>Here we can see that for each key, the library assigned a type and added an example, which will be useful later. Also, we can see how the function addresses arrays — it iterates over all the elements of the array and reduces them to a single schema element.</p><p><strong>Note: Although this example involves an awfully flat object, the library copes with any amount of nesting. Check the </strong><a href="https://github.com/intelligogroup/object-to-schema-mapper"><strong>github repository</strong></a><strong> for more information.</strong></p><p>The problem is, this API is not my primary source of data and since there are several sources, I have my in-house schema which I’m working with, that looks like this:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/2aff2eb36164057dba34bfe2f11e511d/href">https://medium.com/media/2aff2eb36164057dba34bfe2f11e511d/href</a></iframe><p>3. Let’s write a configuration file for the mapper function to transform the response to the wanted schema. The configuration is typed:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7517176a7094614f9d72c64d3f2cc9bd/href">https://medium.com/media/7517176a7094614f9d72c64d3f2cc9bd/href</a></iframe><p>Let’s break down the configuration:</p><p><strong>source:</strong> The path of the key that you want to transform.</p><p><strong>target</strong>:</p><ul><li><strong>path</strong>: Similar to the source parameter, but now it indicates the path of the key for the mapper to create.</li><li><strong>defaultValue</strong>: If no value present in the source object, the default value will be used.</li><li><strong>conditionalValue</strong>: targetPathCondition<strong> </strong>is the path in the target object, that will be checked for presence of a value. If the value exists,value will be inserted to the <strong>path</strong> that was specified.</li><li><strong>priority</strong>: You can map several sources to the same target path. By using <strong>priority </strong>to control what data to use when more than one mapping retrieves data from the source object.</li><li><strong>predefinedTransformations</strong>: Lets you use defined functions to transform data in simple ways. Currently existing functions: “toUpperCase”, “toLowerCase”, “titleCase”, “‘toDate”, “stringToArray” and “arrayToString”. The arrayToString is only intended for string[] We’ll see how to use these functions later.</li></ul><p>With this information at hand, we can construct our own transformation configuration between the imaginary API and our in-house schema and apply the function mapObject to both the API response and the transformations.</p><p><strong>Note: I’ve added TypeScript types because It’s easier to validate the configuration that way. At our company, we have developed a web-application to easily construct such configuration.</strong></p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/358dca62d72c2e1b865071562707880e/href">https://medium.com/media/358dca62d72c2e1b865071562707880e/href</a></iframe><ul><li>You can see that some transformations were simple and don’t really require such library as this, like the title transformation which only needed to change the key.</li><li>The most powerful use case is when confronting data nested in arrays. This library was initially written because I couldn’t find any proper tool in the NPM ecosystem which could achieve such feats. We can see the example in the ratings and date.release transformations, where the library visits each element inside an array and maps it to its dedicated target path. Note that when the source path is nested in arrays, like key[].key2[].key3[].key4 Which means that the object looks like this:</li></ul><pre>{<br>  &quot;key&quot;: [<br>    &quot;key2&quot;: [<br>      &quot;key3&quot;: [</pre><pre>        { &quot;key4&quot;: &lt;&lt;primitive or object&gt;&gt; },<br>        { &quot;key4&quot;: &lt;&lt;primitive or object&gt;&gt; }, // and so on</pre><pre>      ]<br>    ]<br>  ]<br>}</pre><p>If you want to map each element of key3 to some other target path, you’ll have to address all nested arrays in the middle. So the target will <strong>have </strong>to look like: otherKey[].otherKey2[].otherKey3[].otherKey4. If you think about it you’ll understand that these rules have to be followed, because each and every array in the middle can have multiple elements, and the library needs to iterate over them, so the target structure has to stay the same.</p><ul><li>You can see the example of a conditional value in the isReleased transformation. Here, we don’t have a source path since the value is generated by looking at other mapped values. I conditioned the isReleased value to be true only if the mapped released value exists.</li><li>An important thing to know, is that all transformations that are applied to an object are optional. That is to say, that if a source object does not meet one or any transformation, no errors will be raised. This lets us cope with APIs that may return partial information.</li></ul><p>Now, let’s see the output of the mapper function:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/b82fc32a2ca1a834d929ed27da67a862/href">https://medium.com/media/b82fc32a2ca1a834d929ed27da67a862/href</a></iframe><p>As you can see, the response was successfully mapped to our in-house schema.</p><p>4. Le’t get back to that example key, seen after schema creation (2). Let’s say you’ve extracted the schema from a result of an API. Now all you have to do when creating/updating the mapping configuration (transformations) is retrieve the API schema that was created using this library and use a function that extracts a sample object from the schema. That way, you don’t have to call the API again to get a raw response object:</p><pre>import response from &#39;./response.json&#39;;<br>import { mapObjectToSchema, extractExampleFromSchema, } from &#39;@intelligo.ai/object-to-schema&#39;;</pre><pre>const schema = mapObjectToSchema(response); // or retrieve from DB<br>const almostOriginalObject = extractExampleFromSchema(schema);</pre><pre>console.log(JSON.stringify(almostOriginalObject, null, 2));</pre><p>You can see the output:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/bf68a5479007de4b1499cc7f8e9ac87f/href">https://medium.com/media/bf68a5479007de4b1499cc7f8e9ac87f/href</a></iframe><p>If you watch carefully, you can see the difference between the original response and the reverse-extracted response. Arrays were reduced to a single-element arrays. These single-element arrays contain all possible keys from the multi-element original array.</p><p>Now, you can test your mapping over this pseudo-response object and see whether your configurations are correct.</p><p>5. There are a lot more things you can do with this library. For example we, at <a href="https://www.linkedin.com/company/intelligo-group/">Intelligo-Group</a> created an endpoint to update the schema of a particular API and we actually take between hundreds to thousands of raw responses that were saved in our system and pipe them through the schema creator function which can actually accept two arguments. This is the function signature:</p><pre>function mapObjectToSchema(obj: SomeObj, schema: SomeObj = {}): SomeObj</pre><p>The second argument is an already existing schema, which will be enriched by the new response object that is received in the form of the first argument.</p><p>Also, we have created a UI interface web-app (which is currently not an open source application) to easily map between the response schema and our in-house schema. All it does is output the configuration in the format of the above mentioned Transform[].</p><p><em>Thanks for reading and I hope you’ll enjoy the library as much as we enjoy it at </em><a href="https://www.linkedin.com/company/intelligo-group/"><em>Intelligo-Group</em></a><em>.</em></p><p><em>NPM: </em><a href="https://www.npmjs.com/package/@intelligo.ai/object-to-schema">https://www.npmjs.com/package/@intelligo.ai/object-to-schema</a></p><p><em>Github: </em><a href="https://github.com/intelligogroup/object-to-schema-mapper">https://github.com/intelligogroup/object-to-schema-mapper</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=d8e878d2ed94" width="1" height="1" alt=""><hr><p><a href="https://levelup.gitconnected.com/working-with-schemas-in-a-schemaless-ish-world-d8e878d2ed94">Working with schemas in a schemaless-ish world</a> was originally published in <a href="https://levelup.gitconnected.com/">Level Up Coding</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>