<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Equeum on Medium]]></title>
        <description><![CDATA[Stories by Equeum on Medium]]></description>
        <link>https://medium.com/@equeum?source=rss-60de9b5d3419------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/2*Xf5tJ9No8Nmfd4juuUyEtQ.jpeg</url>
            <title>Stories by Equeum on Medium</title>
            <link>https://medium.com/@equeum?source=rss-60de9b5d3419------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Sun, 22 Nov 2026 18:23:38 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@equeum/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[Move Over, Big Data: Wide Data is the Next Big Thing]]></title>
            <link>https://medium.com/@equeum/move-over-big-data-wide-data-is-the-next-big-thing-318625c7069d?source=rss-60de9b5d3419------2</link>
            <guid isPermaLink="false">https://medium.com/p/318625c7069d</guid>
            <category><![CDATA[data]]></category>
            <category><![CDATA[data-science]]></category>
            <dc:creator><![CDATA[Equeum]]></dc:creator>
            <pubDate>Mon, 13 Apr 2020 04:37:01 GMT</pubDate>
            <atom:updated>2020-04-13T04:37:01.861Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7wN5t9ILU0fpnhbMX2vtng.jpeg" /></figure><p>The guiding principle of big data is that all data is useful, and therefore all data should be collected. But the amount of data we have created has grown exponentially in recent years: according to <a href="https://techjury.net/stats-about/big-data-statistics/">EMC</a>, in 2020 there will be around 40 zettabytes — 40 trillion gigabytes — of data. And since big data cannot be stored or processed using existing database or software techniques, big data collection often requires a significant investment in infrastructure. Big data can be an overwhelming undertaking, but fortunately, there’s a new way for users to find answers: wide data.</p><p>What is wide data? Simply put, it’s a different approach to data analysis that looks for commonalities that tie together data sets from disparate sources. Big data focuses on discrete data sets, but the sheer volume of information makes it difficult to glean meaningful and timely insights from them. In fact, the drive to collect every possible piece of data can make it difficult to figure out how to use big data — a 2016 report from <a href="https://www.mckinsey.com/business-functions/mckinsey-digital/our-insights/straight-talk-about-big-data">McKinsey</a> found that only 1% of the world’s data has actually been analyzed. The wide data approach values quality over quantity, collecting smaller and more manageable fragments in a variety of different formats.</p><p>To put it another way, if you need help with foot pain, you’ll probably want to see a podiatrist. But if you want a complete checkup, you’d be better off going to your regular doctor. The same concept applies when it comes to data: the value of big data is in the user’s ability to gain a thorough understanding of one specific, individualized data set. But data is becoming more varied and fragmented, and the number of different databases for each of these individual data sets has more than doubled. Data is more segmented than ever, so any information gathered with big data will be more siloed, which further diminishes the utility of big data.</p><p>That’s what makes wide data so valuable: by trading depth of information for breadth of information, users can work across and between varied, fragmented data sets to pull out useful insights from data and find practical applications for that information. And because wide data doesn’t require sifting through mountains of information, users have more time to find correlations that might otherwise have been overlooked.</p><p>There are opportunities for individuals and groups in many different industries to begin using wide data, including in finance. The role of big data in finance has expanded considerably over the past decade, helping firms identify customer needs and market trends. But the most significant impact big data has had on the finance industry is its use in algorithmic trading.</p><p>The idea behind algorithmic trading is simple: a program uses a mathematical model to execute trades at the best possible prices, while also reducing the potential for human error. Big data made it easier to incorporate more factors into that mathematical model — for example, the complete price history of a given stock — in order to help the program more accurately decide the appropriate time to execute a trade. This additional accuracy allowed firms to start automating the trading process and contributed to the rise of “robo-advisors.”</p><p>However, the way big data is structured places a ceiling on how accurate those automated programs can be. Remember, big data is siloed, which means the program’s estimate for when to execute a trade is based solely on that one stock’s historical price. But stock prices don’t exist in a vacuum; there are many external factors that can cause the price to rise or fall, such as how competitors’ stocks are performing, overall market conditions, trade deals, consumer confidence, and so on.</p><p>The ability to account for these external factors is where wide data clearly outshines big data for investment management. Incorporating data from new sources allows software developers to fine-tune their algorithmic trading models, leading to improved quality of automated trades for investment firms. And because wide data is more scalable, firms can continue to incorporate new data into their algorithms as it becomes available, allowing firms to provide greater value to clients without the need for costly infrastructure expansion.</p><p>Big data helped revolutionize the way we process and use information, but big data is only useful when it can be properly analyzed. Otherwise, it simply creates piles of information that grow less actionable by the day. And if individuals and organizations struggled to extract value from big data before, they will almost certainly be overwhelmed in the near future: <a href="https://www.seagate.com/files/www-content/our-story/trends/files/idc-seagate-dataage-whitepaper.pdf">IDC</a> predicts that by 2025, the amount of data created will grow from 40 zettabytes to 175 zettabytes. Ironically enough, there is simply too much data for big data to be effective.</p><p>As any data scientist knows, having the right data is more important than having the most data. Wide data is flexible, easy to process and store, and can be used in a variety of different ways to find the right solution. With all these advantages in wide data’s favor, one thing seems certain: big data’s days are numbered.</p><p>Join us to fuel the data economy: <a href="https://equeum.com/">https://equeum.com/</a></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=318625c7069d" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Brief History of Forecasting]]></title>
            <link>https://medium.com/@equeum/a-brief-history-of-forecasting-356f5fba847b?source=rss-60de9b5d3419------2</link>
            <guid isPermaLink="false">https://medium.com/p/356f5fba847b</guid>
            <category><![CDATA[statistics]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[forecasting]]></category>
            <category><![CDATA[time-series-data]]></category>
            <category><![CDATA[time-series-forecasting]]></category>
            <dc:creator><![CDATA[Equeum]]></dc:creator>
            <pubDate>Sat, 19 Oct 2019 03:23:18 GMT</pubDate>
            <atom:updated>2019-10-21T14:06:21.036Z</atom:updated>
            <content:encoded><![CDATA[<h3>Time Series Competitions: Forecasting History</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*4ld4910NAmglBpnfR-O_PQ.jpeg" /></figure><p>Time series forecasting problems are a bane of general statistics. On the one hand, many of the interesting problems of the human condition are represented by data in a temporal context: at some level all data is time series data, so it’s not surprising that solutions to complex forecasting problems are sought in fields as diverse as finance, security, and IoT. On the other hand, time series data presents problems that make it fraught: specifically, time series data violates a cardinal principle of most classical statistics, which depends on independent observations. Observations in a series are by definition not independent, creating all sorts of application specific modifications to standard methodology.</p><p>Time series forecasting has gone through a long evolution as a discipline in pursuit of an answer to an ethological question of data science. The question was about the purpose of modelling: whether, in the case of processes producing time series data, there is some particular underlying process- and if so, whether the modeler’s job is to find it.</p><p>This direction led the field for a time to believe that, yes! It was assumed that time series data could be accurately modeled, leading to increasingly complex models produced with questionable results.</p><h3>Forecasting Competitions: Simpler is Better</h3><p>In 1979, Spyros Makridakis and Michèle Hibon put together a collection of 111 time series and compared many more forecasting methods in a paper. Some professors were adamant that there must be a true model waiting to be discovered, although Makridakis and Hibon argued that empirical evidence disagrees with their theoretical arguments. Makridakis and Hibon’s findings claimed that forecast accuracy was a feature of time horizon and evaluation metrics, and that simpler methods tend to outperform complex methods over longer periods. The competition founders’ competency was criticized and questioned, leading them to hold the M1 — a competition focused on time series predictions.</p><p>The first official Makridakis competition, in 1982, involved 1,001 series and was the first true forecasting competition. Anyone could submit forecasts and multiple forecast measures could be used to determine the most accurate method. The time series were from demography, industry, and economics, and ranged in length between 9 and 132 observations. All the data were either annual, quarterly or monthly. This competition confirmed the findings of their 1979 paper:</p><ul><li>Statistically sophisticated or complex methods do not typically produce more accurate forecasts than simpler ones.</li><li>The ranking of the performance of the various methods varies according to the accuracy measure being used.</li><li>The accuracy of the combination of various methods outperforms, on average, the individual methods being combined, and does well in comparison with other methods.</li><li>The performance of the various methods depends on the length of the forecasting horizon.</li></ul><p>The findings of this competition pushed future researchers to focus on models that produce good forecasts, rather than their mathematical properties. They were also made aware of the dangers of overfitting and started considering automating forecasting methods. These were very new findings that had a profound impact on research in the field.</p><h3>State of the Art in Time Series Forecasting</h3><p>Since then, the Makridakis competitions have been held occasionally to define the state of the art in time series forecasting. While the M2 didn’t produce particular findings, the M3 threw their conclusions about simplicity into question for the first time: ForecastPro performed extremely well, which selects between ES and ARIMA methods based on a state space approximation.</p><p>The M3 demonstrated that novel modelling methods generating more complex forecasting models, which may normally produce overfit, were catching up to classical simple stats. The statement Makridakis and Hibon made after the M1 could be qualified to state that simple models are always better for predicting particular known processes, but learning mechanisms are useful for finding a generalized best fit solution. A hybrid approach can give the best of both worlds, avoiding overfit while allowing series specificity.</p><p>The most recent competition, the M4, ended on May 31, 2018. The M4 extended previous set ups in depth, breadth, and reproducibility, reflecting modern day “Big Data” challenges. The M4 included 100,000 time series with weekly, daily and hourly data, along with annual, quarterly and monthly data. The main objective of the M4 was to “‘learn how to improve the forecasting accuracy, and how such learning can be applied to advance the theory and practice of forecasting”.</p><p>The winning method : a hybrid approach that utilized both statistical and machine learning features (from one of Uber’s leading data scientists, Slawek Smyl). The second most accurate submission was also a combination of seven statistical methods and a machine learning approach. These two “combination” methods achieved amazing success, specifying the 95% prediction intervals correctly and taking advantage of the ability of machine learning based models to account for non-linearities in the data.</p><p>The improvement in performance is gained by combining the strengths of statistical approaches to time series forecasting. Machine learning models allow generalization over a very large sample of data, and specify observations very easily. This allows the model to account for lack of observation independence - a pitfall for pure machine learning approaches. The winning model leveraged a hierarchical approach, which generated a RNN “base” for the forecasts, and specified it with parameters to an ES function that were optimized for each unique series. These advancements were only made possible by certain advances in the state of the art: compute power and recent innovations in GNNs, which allowed for the formation of hierarchical model architectures.</p><p>Six pure machine learning methods performed poorly, with none of them being more accurate than the combination benchmark. Elsewhere in the universe, a Canadian computer scientist claims to have solved the univariate time series problem with a pure deep learning approach, beating the bench by 11% and managing to avoid the necessity for time series specific components completely. But this isn’t surprising- machine learning is just another tool in the statistician’s toolbox, and requires good design to deliver results.</p><p><em>Equeum is the platform enabling collective intelligence in capital markets research. We are bringing together data owners, developers, and asset managers to re-imagine the supply chain fueling Wall Street. </em><a href="https://equeum.com/"><em>Visit us here</em></a><em> to join the mission- we’re eager to have you be a part of the journey.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=356f5fba847b" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Collective Intelligence in Finance: Your Guide to Participating in Challenge Driven Innovation]]></title>
            <link>https://medium.com/@equeum/collective-intelligence-in-finance-your-guide-to-participating-in-challenge-driven-innovation-60a9e4013d28?source=rss-60de9b5d3419------2</link>
            <guid isPermaLink="false">https://medium.com/p/60a9e4013d28</guid>
            <category><![CDATA[fintech]]></category>
            <category><![CDATA[collective-intelligence]]></category>
            <category><![CDATA[quant]]></category>
            <category><![CDATA[crowdsourcing]]></category>
            <dc:creator><![CDATA[Equeum]]></dc:creator>
            <pubDate>Mon, 07 Oct 2019 17:24:23 GMT</pubDate>
            <atom:updated>2019-10-07T17:24:23.049Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*3dzr2R6ONcSpLEpjU5X_pQ.png" /></figure><p>Open markets provide a rich landscape for innovation by building on the best ideas produced by a community. Decentralized networks replace top-down product development with collective intelligence directed at a wicked problem: an open ecosystem means benefiting from the creativity of a global talent pool, with individuals bringing unique experiences and perspectives to the solution.</p><p>Capital markets research has been dipping its toes into leveraging the power of the crowd in a quest for a more efficient innovation strategy: from research aggregation to alpha generation, prediction tournaments, open source projects, learning platforms, and more, industry titans and startups alike have been seeking the best way to integrate the heavy security demands of financial services with the innovative potential of open communities.</p><p>Every flavor of open network opens participants to something a little different. While community led projects range from offering learning opportunities to intellectual challenges or even workplace enablement, many also offer resources for the consumption of their communities to incent participation. Other projects act as a hub for low-filter research aggregation, or try to enter symbiosis with developers by offering otherwise rare data under a freemium model with a monetization strategy for commercial endeavors.</p><p>These ten platforms leverage collective intelligence to drive results in capital markets research in different ways. Whether you’re looking to learn a skill, develop a trading strategy for personal use, compete against the best and brightest, or market a product, open markets can provide a way to engage with a space that is otherwise hard to enter.</p><h3>Industry Projects</h3><p><a href="https://seekingalpha.com/"><strong>Seeking Alpha</strong></a></p><p>What is it: Crowd-sourced market research</p><p>What does it offer:</p><ul><li>Market research</li><li>Paid access to private communities with investor focus</li></ul><p>Cost of participation: Free (Premium account for minimum $19.99/month)</p><p>Seeking Alpha is a crowd-sourced content service for financial markets, connecting millions of investors to share ideas and make informed investment decisions. Articles and research published in the community cover a broad range of stocks, asset classes, ETFs and investment strategies. Investment theses undergo rigorous editorial reviews, ensuring the highest quality standards. In contrast to other equity research platforms, insight is provided by the contributor base of investors and industry experts rather than the sell side, and more than 10,000 investing ideas are published each month. Through crowdsourcing, Seeking Alpha allows people to connect and share ideas, news, debate amongst themselves, and make better informed investing decisions. Advanced charting, data visualizations, and technical and fundamental analysis add clarity and enable informed, data-driven decision-making.</p><p><a href="https://www.refinitiv.com/en/products/intelligent-tagging-text-analytics"><strong>Open Calais</strong></a></p><p>What is it: A RESTful API for named entity recognition and semantic mapping of unstructured data</p><p>What does it offer:</p><ul><li>A limited free tier to the Refinitiv Intelligent Tagging API</li><li>Extraction of entities [companies and people] and relations [events like IPOs, product recalls, or reorganizations]</li><li>Classification into industries, topics, and social tags with relevance scoring</li></ul><p>Cost of participation:</p><ul><li>For free, users can get a “free-to-experiment model” to begin API integration</li><li>Users must pay for a fully managed model, in-house model, or an environment in their own Azure account</li></ul><p>Open Calais is a free tier of the Intelligent Tagging API, which turns unstructured data into intelligent information by automatically creating rich semantic metadata for news articles, research, emails, and other unstructured text formats. Within a second, Calais can analyze documents to find the entities within it, as well as facts and events that might be hidden within the text. Information is extracted in RDF triples, and can be integrated into sentiment based systematic trading strategies making it a great tool for the novice financial software engineer.</p><p><a href="https://www.finos.org/"><strong>FINOS</strong></a></p><p>What is it: Industry-organized collective</p><p>What does it offer: Resources and organizational structure for applications geared to technology enablement in financial services.</p><p>Cost of participation: Fee for membership benefits</p><p>FINOS is a non profit organization that provides an independent setting to deliver software and standards that address common industry challenges and drive innovation. Participants develop open source solutions with the intent of fostering collaborative culture, increasing system interoperability, and lowering costs of integration among disparate legacy systems. Unlike most projects on this list, FINOS is explicitly geared to enabling knowledge workers in financial services, and moving the industry forward: projects are developed collaboratively by representatives of various stakeholders under a strict governance structure.</p><h3>Community Projects</h3><p><a href="https://www.firmai.org/"><strong>FirmAI</strong></a></p><p>What is it: Crowdsourced code repository</p><p>What does it offer: Free business intelligence tools for research, data analysis, and other domain specific applications.</p><p>Cost of participation: Free</p><p>FirmAI is a repository of experimental business intelligence tools (BITs) developed as independent projects by members of GitHub. While most BITs are used in corporate and enterprise settings, FirmAI seeks to develop an offering at a capability and price point accessible to small businesses. Derek Snow of FirmAI is changing the world by offering intelligent business automation to anyone willing to spend some time figuring things out. The website also offers a wealth of knowledge, tutorials, examples and tooling, making it a great starting point to learn practical data science.</p><p><a href="https://www.quantconnect.com/"><strong>QuantConnect</strong></a></p><p>What is it: Research and monetization platform</p><p>What does it offer:</p><ul><li>Resources to develop algorithms, as well as a platform to market, run, and monetize the end product</li><li>Participants develop algorithmic trading systems using data provided through the platform</li><li>LEAN — python package</li><li>Monetization via algo licensing</li></ul><p>Cost of participation: Free</p><p>QuantConnect is an open-source, cloud-based algorithmic trading platform for equities, FX, futures, options, derivatives and cryptocurrencies. It enables the quick design of algorithmic trading systems by providing market data and a cluster computer directly to engineers around the world. Anyone can create a live trading algorithm for personal use, or for licensing to the platform’s base of interested buy side firms and investors. While algorithms often need to demonstrate over three months of history for consideration, making it poorly suited to commercial activity, QuantConnect is a well resourced community that can provide participants with a great learning experience.</p><p><a href="https://www.openquants.com/"><strong>OpenQuants</strong></a></p><p>What is it: Educational website</p><p>What does it offer: Education in quantitative analysis, including resources for financial data science, usage of alternative data, and practical methodology.</p><p>Cost of participation: Free or minimum $5/month on Patreon</p><p>OpenQuants.com creates free, reproducible and open-source quantitative research in financial data science and alternative data including articles, interviews, and a live, open book. They emphasize openness and reproducibility, and work towards opening access to the resources and skill sets required in modern quantitative modelling. Some articles and interviews are free, but other features are available only to supporters on Patreon. Their Open Quant Live Book initiative is open to co-authors and contributors, and aims to be an introduction to the most important aspects of financial data analysis, algo trading, portfolio selection, econophysics, and machine learning in finance.</p><p><a href="https://quantocracy.com/"><strong>Quantocracy</strong></a></p><p>What is it: Research feed</p><p>What does it offer: Curated mashup of quant blogs</p><p>Cost of participation: Free</p><p>Quantocracy is an organized mashup of blogs on the topic of quantitative and empirical trading. The site is the number one source of new readers for most blogs on their mashup, while offering subscribers a one stop shop for trending research and industry news.</p><h3>Alternative Hedge Funds</h3><p><a href="https://numer.ai/"><strong>Numerai</strong></a></p><p>What is it: Prediction tournament</p><p>What does it offer: Participants solve machine learning problems as part of a network wide prediction challenge</p><p>Cost of participation: Free (potential to be paid)</p><p>Numerai hosts a weekly data science tournament to direct its hedge fund activity. Participants aim to generate predictions for an un-released validation set, and with the most successful entries earning payouts in NMR, the platform’s utility token. Data scientists work with clean, structured, and obfuscated data, turning the tournament into a difficult feature engineering problem. The best models are incorporated into the Numerai meta-model, which is traded by the fund directly. Numerai has paid out just shy of $8M to its network to date.</p><p><a href="https://www.worldquant.com/home/"><strong>WorldQuant</strong></a></p><p>What is it: Alternative hedge fund</p><p>What does it offer: Investment services and competitions</p><p>Cost of participation: No fee to participate in the competition</p><p>WorldQuant is an international hedge fund and quantitative investment management firm that gives people the opportunity to learn about quantitative finance and compete in their WorldQuant challenge. This year-round competition allows contestants to use the WebSim platform to create algorithmic alphas. WorldQuant opens up internal data to infinite numbers of independent contributors and has even launched their own independent portfolio manager platform.</p><p><a href="https://www.quantopian.com/home"><strong>Quantopian</strong></a></p><p>What is it: Alternative hedge fund</p><p>What does it offer: In addition to a hosted tournament, Quantopian offers educational tools and open source packages.</p><p>Cost of participation: Free</p><p>Quantopian is crowd-sourced hedge fund that allows novice quantitative analysts to test and develop trading algorithms. They also provide data, education, and a research environment to their users, and the opportunity to be paid for algorithms that fit a particular investment profile.</p><p>Unlocking the potential of collective intelligence and open data in capital markets is a problem that gets solved along multiple fronts. In addition to offering a data-enriched research environment and providing the infrastructure to support development without labor duplication, platform stewards need to solve the relatively harder problem of creatively solving for good challenge definition, incentive alignment, and skill development in order to bootstrap the formation of a healthy ecosystem. Explore the communities and offerings above for an introduction to quantitative trading and the existing landscape, and let us know what you think! Where does collective intelligence fit in the capital markets research landscape?</p><p><em>Equeum is the platform enabling collective intelligence in capital markets research. We are bringing together data owners, developers, and asset managers to re-imagine the supply chain fueling Wall Street. </em><a href="https://equeum.com/"><em>Visit us here</em></a><em> to join the mission- we’re eager to have you be a part of the journey.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=60a9e4013d28" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>