<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Tom Malcolmson on Medium]]></title>
        <description><![CDATA[Stories by Tom Malcolmson on Medium]]></description>
        <link>https://medium.com/@tomdot?source=rss-9ca5b6774d6a------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*hCooQmhsLdtwLgW32mKfcQ.jpeg</url>
            <title>Stories by Tom Malcolmson on Medium</title>
            <link>https://medium.com/@tomdot?source=rss-9ca5b6774d6a------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 08:25:40 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tomdot/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com/" rel="hub"/>
        <item>
            <title><![CDATA[FSD now, with Tesla Vision and Teleops]]></title>
            <link>https://medium.com/pressing-progress/fsd-now-with-tesla-vision-and-teleops-a034a70069f4?source=rss-9ca5b6774d6a------2</link>
            <guid isPermaLink="false">https://medium.com/p/a034a70069f4</guid>
            <category><![CDATA[teleoperation]]></category>
            <category><![CDATA[tesla]]></category>
            <category><![CDATA[autonomous-cars]]></category>
            <category><![CDATA[ridesharing]]></category>
            <category><![CDATA[elon-musk]]></category>
            <dc:creator><![CDATA[Tom Malcolmson]]></dc:creator>
            <pubDate>Tue, 25 May 2021 22:33:04 GMT</pubDate>
            <atom:updated>2021-05-25T22:33:04.315Z</atom:updated>
            <content:encoded><![CDATA[<h3>Introduction</h3><blockquote>This article advocates for and predicts the use of teleops (remote drivers) as a fallback for Tesla’s current vision based FSD system, in order to deliver full autonomy much sooner. It makes the case that this isn’t just technically and financially viable but is a good direction for Tesla with significant advantages.</blockquote><blockquote>By the way, to try to make this a quicker read, we’ve marked sections that can be skipped (like sidebars, but better for small screens) by starting them with “by the way”, and we’ve bolded key parts.</blockquote><p>FSD is “Full Self Driving” and refers both to a feature that Tesla already sells for its vehicle and to their vision of fully autonomous vehicles. The gulf between these is unfortunately still large and is quite controversial, but the gulf could be bridged and that vision could be enabled now using teleops (remote control) in combination with their existing Tesla Vision based system.</p><h3>The Potential for Progress is Immense</h3><p><strong>Autonomous vehicles hold incredible promise, to save millions of lives, to free up countless driver hours, to transform transportation and cities, and to reduce service costs (eg. for deliveries). </strong>Given the theme of this publication (Press Progress) it will be no surprise to hear that we feel strongly that delaying this revolution unduly would be a tragedy, and we appreciate that Tesla and Musk have pushed forward with their vision rather than giving in to the nonsensical criteria thrown up to slow it down, such as the idea that a single death is too much in spite of the huge number of lives that could be saved — this sounds harsh, but it must be accepted that autonomous vehicles will cause deaths in order to unlock the vastly greater number of lives saved.</p><p>Musk referenced some of the potential in the <a href="https://www.fool.com/earnings/call-transcripts/2021/04/27/tesla-tsla-q1-2021-earnings-call-transcript/">Q1&#39;21 Earnings Call</a>…</p><blockquote>“And so if you have a system which has very good eyes, you can see in all directions at once, you can see three focal points ahead or forward, but it never gets tired. It’s never sort of texting. It has redundancy and its reaction time is super human. Then it seems pretty obvious that such a system would achieve an extremely high level of safety, far in excess of the average person.”</blockquote><p><em>By the way</em>, we have a personal perspective on this safety discussion. 1.4 million people die in car accidents every year, but even that isn’t the whole story. Ten years ago Nella, who is otherwise healthy and fit, was involved in a car accident. She and the other driver both walked away from the accident and do not show up in any statistics of fatalities or injuries, and yet she is plagued by back pain to this day. So, as consequential as that 1.4 million death figure is, we think it is just the tip of the iceberg of the harm caused by car accidents.</p><h3>The Problem</h3><p><strong>Tesla FSD is not near to achieving level 4 (L4) autonomy — the minimum level that would be necessary to realize these dreams.</strong></p><p><em>By the way</em>, these “levels” of autonomy are defined, roughly, as follows. (For more, <a href="https://google.com/search?q=SAE+J3016">search on SAE J3016</a>; the official document is <a href="https://www.sae.org/standards/content/j3016_202104">here</a>.)</p><ul><li>L2 — Tesla autopilot (an advanced driver assistance systems)</li><li>L3 — Tesla “feature complete” FSD of 2021 (conditional automation, requiring human supervision)</li><li>L4 — Full autonomy (no driver required) in certain locations and weather conditions. Waymo’s service, which has no safety driver, but only operates in “easy” locations and can be impacted by adverse weather conditions, is an example of this.</li><li>L5 — Full autonomy under any conditions — no one is close to this or is even on the path to this (consider the traffic in the most chaotic city you’ve visited!).</li></ul><p>We know that many readers who only have time to follow the area casually have very different expectations, but I’m not aware of any serious analysis with projections that match the statements of Musk or the expectations of many in the Tesla community (see the ‘analysts’ sidebar).<br> <br>You may judge that your Tesla vehicle seems to be very close, but the experience of the most advanced players in this area (Waymo and Cruise) shows us that once you get extremely close to solving the problem, you still have years to go before you get to 100%. Tesla’s data is a great strategic advantage in several areas, but it isn’t the magic that some of the panderers make it out to be. (Perhaps you think we are just advocating for the adoption of a different sort of magic here — if so, may the best magic win!)</p><p><em>By the way</em>, to put a positive bound on the analysts’ projections consider ARK Invest, which is an investment management firm that is one of the most bullish and high-profile Tesla supporters (and who have based much of their pro-Tesla argument on the potential of Tesla’s FSD). They <a href="https://ark-invest.com/articles/analyst-research/tesla-price-target-2/">recently declared</a> that they were upping their <strong>estimated odds of Tesla achieving fully autonomous driving by 2025 to 50%. Since then ARK have taken note of Tesla’s decision to stop augmenting their vision system with radar, and predicted that this change will delay that timeline</strong>.</p><p><strong>This leaves Tesla in a real predicament, with huge unrealized potential, unfulfilled promises and even </strong><a href="https://www.reddit.com/r/teslamotors/comments/mbetme/serious_what_is_teslas_exposure_if_fsd_doesnt/"><strong>liability concerns</strong></a><strong>. </strong>As well, there is growing concern about the dangers of being in this ‘not quite finished’ state, with many customers overconfident in its abilities. We believe that many of the incidents trumpeted in the media are the result of drivers being irresponsible to a degree that could cause trouble in any type of car, and that even in its current state Tesla’s autopilot/FSD are already preventing accidents. On the other hand, the number of accidents it is preventing now is a small fraction of what it will prevent once it is mature, and there is also a legitimate concern about what happens when you give customers a product with incredible potential that is close to being realized particularly when, for many of them, it falls short of what they understand it to be; this could well become a bigger problem as it gets closer to being finished.</p><p><em>By the way,</em> for those interested in TSLA (Tesla’s stock) it is important to note that the bull case centres around autonomy/FSD: selling it to customers and using it to enable the Tesla ride-hailing network. Competitors such as Waymo (a sister company of Google) and Baidu in China have already launched their own small, commercial robotaxi services, and may add their robotaxi’s to existing ride-hailing networks. If Tesla could realize their robotaxi service soon it would easily eclipse these potential competitors who have (by necessity) started small and slow. Waymo has shown how difficult and expensive it can be to ramp up such a service even with great technology. Tesla could leapfrog them easily, but if they have to wait several years then it’s a very different story. At that point, the incremental cost of their lidar based solutions will be minimal, they will have ramped up, and safety may become the dominant consideration for a public that seems to get freaked out by a single accident if it involves new technology. We would suggest that, in deciding whether to buy or sell TSLA the most important question is what are the prospects for them moving from L2/L3 to L4/L5 autonomy.</p><h4>What’s holding Tesla back?</h4><p><strong>The main problem is edge cases. </strong>One can view endless videos on Youtube that make it appear that Tesla has solved autonomy because they aren’t showing you the edge cases. Even once the ‘feature complete’ version of their FSD is working well (what Tesla is working towards this year), the experience of their competitors in this space have shown that the edge cases are extremely hard to address.</p><h3>The Solution — Tesla Vision plus Teleops</h3><p><strong>This problem can be resolved by complementing Tesla Vision with teleops, used only as required for edge cases. </strong>Teleops is short for Teleoperation (or remote operation), which is human operators remotely controlling the vehicle.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/984/1*oJJ71V8zLHLOrfl9E6O2gg.jpeg" /></figure><p>When an edge case arises, eg. a difficult intersection during rush hour or a strip of a road in front of some bars at night, the vehicle uses its cellular connection to relay some of its video feeds to a human operator who temporarily assists in driving the vehicle — this can range from taking complete control to just assisting in the interpretation of the video.</p><p>Note that the onboard FSD will always be on and available — it will just be in something like shadow mode (the mode currently used when the human driver is in control of the vehicle) if a remote operator is actively driving the vehicle, but doing some additional processing geared mainly towards minimizing the amount of data that needs to be sent.</p><p><em>By the way, t</em>eleops is already widely employed, eg. for piloting drones, and is even already used in a way very much like I’m proposing here, by Voyage, Inc.. They developed a successful (technically and operationally, at least — not sure about commercially) business around robotaxis in large retirement communities and towns, but their aspirations were always that this should be a stepping stone to a regular (full speed) service. Last year they introduced teleops to augment their vehicle’s onboard autonomous driving technology ( for more see <a href="https://news.voyage.auto/introducing-voyage-telessist-a085e4c1f691">Introducing Voyage Telessist</a>). They have since been purchased by Cruise, and we don’t know what plans Cruise has for Voyage’s teleops, but using teleops in this manner actually makes considerably more sense for Tesla than for Voyage or Cruise since Tesla’s sensor network is vision only.</p><p>The Tesla Vision + teleops solution would be most useful for <strong>location + time based edge cases</strong>: driving segments that involve difficult situations or uncertainty, usually at particular times of day. Where possible, routing will avoid these (a good idea from a safety perspective anyway) but where that is impossible a remote driver who is experienced in the scenario at hand can be brought online seconds ahead of time. If there is a communications issue then the FSD takes control and goes into a “caution” mode, behaving as conservatively as possible and possibly allowing the sort of decelerations that autonomy engineers are trying to avoid and possibly even pulling over to the side of the road. Obviously this must be made as rare as possible and we explain below why this will in-fact be less common than you think.</p><p>So the onboard FSD is acting as a fallback to the “teleops enhanced” FSD which is a fallback to the onboard FSD — so let’s just say they complement each other very well (we don’t think teleops for cars even makes sense without some onboard autonomy).</p><p>For other types of edge cases, most importantly, <strong>sudden and unexpected events</strong>, there would be a delay before the remote operator became available during which the on board FSD would again be in a “caution” mode. The remote operators would need to be skilled at orienting themselves extremely quickly, but the skill exhibited by video gamers suggests that many people are remarkably good at this, and scale would allow a degree of scenario specialization that would make this easier, eg. an operator could specialize in handling roundabouts, flat tires or snowy conditions.</p><p>Nevertheless, teleops would be less effective at this type of edge case (compared to the location based ones) and Tesla would have to focus early efforts on training the caution mode. These events should be much less common then the others (that’s part of why they are unexpected) but this is important so we will come back to it later.</p><p>As the onboard FSD becomes mature, eliminates the sorts of edge cases discussed, and approaches the sort of complete L4 system that the other vendors are working towards now (with their more sophisticated and expensive sensor systems), there will still be the issue of the large area between L4 and L5 which includes more challenging weather conditions and locations. Here again, the addition of teleops as a complement to Tesla’s onboard FSD will give them a significant advantage.</p><p><em>By the way, </em>let us consider one more significant upside of going this route… once Tesla vehicles are uploading data and getting back driving instructions to augment their FSD, all in real-time, this opens up new possibilities: the server could resolve some of the situations without resorting to human operators, eg. using what it knows from other Tesla vehicles in the area and by applying additional computational resources. This capability would be improved with time and with increasing density of Tesla vehicles, and one would expect that the areas where they offer their ride-hailing service would have lots of Tesla’s, and the places where FSD needed to resort to “outside help” would usually be dense urban areas with high vehicle density. we would expect that this is a long-term end goal for most of the autonomous driving systems anyway: that they evolve from a bunch of vehicles making their own decisions to a network of connected vehicles with a degree of collective intelligence (mediated through the Tesla motherships that is — peer-to-peer vehicle communication is a whole other topic).</p><h3>Possible Issues</h3><p>So then why would Tesla <em>not</em> pursue this solution?</p><p>This hybrid solution doesn’t have the technical “purity” of achieving the holy grail: an L4 autonomous system based purely on Tesla Vision — some might see it as a costly detour on the road to that goal. And yes, it would certainly involve extra cost and complexity relative to the purely onboard FSD system based purely on Tesla Vision, so it would not even be worth pursuing if it weren’t needed. If Tesla really only needs another year to get there then this hybrid solution would not be worth adopting, but we think we’ve been clear that we don’t believe that is the case and this pessimism is very much in line with industry consensus which is that they are many years away from even L4.</p><p>Some might see telops as Elon Musk swallowing his pride and admitting that the vision he has espoused and predicted is not possible in the near term. We don’t see it that way at all. Musk is about making things happen (which is often at odds with technical purity anyway) and it would be quite understandable that Tesla would not have revealed that they had an interim solution such as this up their sleeves. Frankly, it would put all of Musk’s predictions in a much better light.</p><p><em>By the way</em>, Waymo has taken pains to make it clear that while their ‘driver’ (their autonomous driving system) may have to rely on human intervention for strategic planning, they are not using teleops to drive the vehicle so their system should be considered fully autonomous — a semantic and pedantic distinction that we don’t think is even valid. If this distinction were for the regulators then they wouldn’t be exhibiting the same need to get the word out publicly. Perhaps this is pride? Perhaps they fear that teleops would allow competitors to suddenly close the gap that they have gained with such enormous investment.</p><p>All of these points of pride and purity are beside the point, <strong>what matters is whether the solution proposed is viable and can actually solve the problem at hand, so we will now address some of the reasons people might put against that</strong>. We think that cellular networking deficiencies, and costs will be the first that come to mind for many readers, but in fact the latter considerations below are more pertinent.</p><h4>Cellular Communications</h4><p>We think the primary issue that people will bring up to argue against the use of teleops will be the reliability and latency of the cellular communications. I’m not going to get into a detailed, technical discussion of the requirements and capabilities at this moment because <strong>cellular technology is a quickly moving target</strong>.</p><p>In particular, 5G can provide lower latency than 4G and it can allow for <strong>reserving slices of spectrum with defined characteristics</strong> (more <a href="https://spectrum.ieee.org/telecom/wireless/network-slicing-5gs-hottest-feature">here</a>)<strong>, </strong>at difficult locations. AT&amp;T is Tesla’s provider in the U.S. (We focus on the U.S. here because that is likely where they would roll out this service first), and they are rolling out some network slicing in 2021 and they have “plans to expand its 5G footprint into the world of autonomous vehicles” this year. While 5G is often over-hyped, it has key advantages for autonomous vehicles that will make teleops considerably more viable in 2022 then in 2020 when Voyage launched Teleessist.</p><p>The fact that AT&amp;T has plans to cater to autonomous vehicles this year is noteworthy. We think it is likely that all of the autonomous vehicles will want to maintain a real-time, cellular link even if they aren’t using teleops. Waymo is the one we know the most about since it is the only one with a commercial service, and they do indeed fall into this category. And this is not just an interim bandaid — onboard autonomous systems will get better but they are so far from L5 that they will benefit from realtime augmentation for the foreseeable future (forever in my opinion). <strong>Developing this part of the teleops stack — a reliable, low latency, real-time communications link — will be a requirement for autonomous vehicles regardless of whether teleops is being used or not.</strong></p><p>So the feasibility of using cellular is a matter of when and where, rather than if, and further evidence that ‘when’ is soon is that remote controlled robots (as drones, industrial robots, and vehicles) over cellular links are already here — even with onboard autonomy that is inferior to Tesla’s and without 5G network slicing.</p><p><em>By the way</em>, supporting 5G on Tesla vehicles would require upgrading a daughterboard radio (and SIM card) that is located in the MCU (this is based on Model S, but having designed it for simple upgradability we doubt they would have gone backwards on that). Supporting mmWave 5G would be a lot harder but it sounds like AT&amp;T is being the least aggressive of the carriers on that.</p><h4>The Cost</h4><p>The cost is another moving target, with the number of edge cases that require teleops and the cost of cellular data continually decreasing. Every use of teleops will be valuable training data that would improve the fallback performance of the onboard FSD and eventually eliminate the edge case. Also keep in mind that teleops will only be used for a small portion of each trip since the routing software would avoid those tricky intersections, construction zones, busy streets etc. whenever possible.</p><p>The cost of the operators might remain fairly steady initially — increasing as more vehicles start using teleops but decreasing as the number of edge cases remaining decreases.</p><p>The technology will need to be developed and though this isn’t trivial, neither is it cutting edge work like their onboard FSD — after all, in 2020 Voyage, one of the smaller companies in this area — released something very similar to what Tesla would need to do. As noted in the description of the system above, even after Tesla’s onboard FSD realizes the current dream of L4 autonomous driving without teleops, this investment in teleops will remain relevant for many years by helping bridge the gap between L4 and L5 — so this investment in adding teleops will have many years to pay for itself.</p><p>Most importantly, though, this concern about the cost must also be balanced against the revenue it enables, in selling the FSD option to vehicle owners (and avoiding possible class action lawsuits from those who have already paid), in enabling the Tesla robotaxi network, and in the publicity and prestige it would bring to Tesla as the first company to sell true autonomous vehicles to consumers and the only company that can do so at a reasonable price (since their sensor array is so much simpler and cheaper than that of their competitors).</p><p><em>By the way</em>, in spite of these costs, we think that in the long run, as Tesla’s competitors find that they too have to strengthen and optimize the real-time cellular connection with their vehicles because they operate better with some shared intelligence, as they consider how the extra cost and complexity of using lidar and radar as well as vision has left them without the larger fleets and extra revenue that Tesla’s approach has made possible, and as they investigate how to move on from L4 towards L5 autonomous driving, they may wish they had accepted the costs of teleops too.</p><h4>Keeping Sensors Clean</h4><p>Well, this concern is rather mundane compared to the others, but it doesn’t seem to us that Tesla has made adequate provision for keeping the cameras clean in adverse conditions. However, this concern is not specific to this solution — it is a concern about Tesla Vision in general. If an inability to keep the sensors working properly is a deal breaker for teleops then it is a serious problem for FSD and for Tesla in general, and then I’m baffled as to why it would not be the current focus of attention and of incremental hardware improvements.</p><p>Here are two considerations that might ameliorate the problem in the context of their robotaxi service. We anticipate that there would be Tesla employees at their charging stations to clean their robotaxis while they are charging, and they would also clean the cameras. We also expect that Tesla will start their network in locations with consistent good weather.</p><p><em>By the way</em>, as a Canadian this is difficult to face, but to some degree, full autonomy and the ride-hailing network will simply not be available to customers in bad weather in the near future. Obviously this is a major drawback, but we believe Waymo has had to accept this limitation as well — to a smaller degree in Waymo’s case but nevertheless, autonomous driving will be weather dependent for a very long time. This is one of the reasons why robotaxi networks are usually forecast to start as hybrid networks so a decision can be made between autonomous and regular drivers for each individual trip depending on routing possibilities and weather. This could give Tesla an advantage over a competitor such as Waymo if Tesla is prepared to launch a hybrid network (though one avenue Waymo is exploring to address this is to add their robotaxi’s to Lyft’s network, as well as on their own Waymo One network.)</p><h4>Regulatory Issues</h4><p>Regulatory approval is an important consideration and potential impediment for any autonomous driving company or network. Adding teleops to the Tesla technology mix, while it makes Tesla’s technology more complicated, it should not make the regulatory proposition more complicated. After all, the fact that Tesla has now reduced their sensor stack to just cameras doesn’t make it simpler for them to get approval relative to companies that also use lidar and radar. They all must prove that their technology is safe — regardless of what that tech stack comprises — so if adding teleops makes Tesla’s FSD better then it will enhance their potential for regulatory approval, and if it doesn’t then they won’t adopt it.</p><p>Unfortunately, all of the <strong>autonomous driving companies must also manage fickle public perceptions of these new technologies</strong>, and this will influence their chances of regulatory approval. For this, teleops is a bit of an unknown. One of the reasons we are eager to start the discussion about Tesla relying on teleops and to respond to some of the likely concerns, is to address this. Keep in mind though, that the current in-between state of Tesla’s FSD technology has also not been helpful. The quicker they can jump out of this “uncanny valley” between L2 and L4 the better.</p><h4>Technology Mismatch</h4><p>The fundamental proposition here is that teleops, provided when needed rather than continuously, will be able to address most of the edge cases that would otherwise significantly slow down Tesla’s progress. As well, some of the remaining edge cases could be addressed simply by being prioritized over the ones that teleops can handle, and those that remain are rare enough they can be accepted as emergency situations in which the onboard FSD enters a cautious driving mode.</p><p>If, on the other hand, the critical path edge cases are the determination of people’s intentions (which is certainly a challenge for these systems) and this must be done suddenly and in frequently occurring and unexpected locations, then teleops employed in the proposed manner will not help.</p><p>We have speculated and reasoned about the validity of that proposition, and given my opinion, but ultimately only Tesla will be able to make the determination. Obviously we would love to know Tesla’s position on this!</p><h3>Conclusion</h3><p>In conclusion of the discussion about potential issues, keeping the sensors clean and regulatory issues are part of the broader discussion — they are not specific to the technology proposition outlined here. And while we think it will surprise many readers, in my opinion the dependence on cellular communications and the costs are (or soon will be) acceptable, so the viability of the solution proposed here comes down to the technology match — whether teleops is the right complement to address the current limitations of Tesla’s onboard FSD.</p><p>We concede that we have not seen this approach discussed in the media, in spite of the avalanche of Tesla speculation that is created every day (have we missed it?), but neither have I seen anything that would preclude it, or any analysis rejecting it. As far as I can tell, it is simply ignored, and that is why I wrote this.</p><p>Nevertheless, we are confident enough that I’m <strong>predicting that Tesla will adopt teleops in 2022.</strong> The forthcoming ‘feature complete’ version of Tesla’s onboard FSD (now renamed to v9 and likely for Q3 of 2021) is a precondition for the hybrid system to work as we have depicted it here.</p><p>Actually, we had rather <em>assumed</em> that this was Tesla’s plan: that it is one of the reasons they stuck to their simpler (cheaper) sensor stack while still exhibiting such great confidence in the near future potential of their FSD to achieve autonomy. The <a href="https://twitter.com/elonmusk/status/1370522745709158407">recent tweet</a> by Musk (since re-affirmed) about them excluding even radar from the latest incarnation of FSD reinforces this impression. And if they aren’t planning to utilize teleops (and have no other magic in their back pocket) then we consider <a href="https://www.reuters.com/article/us-tesla-autonomous-idUSKBN24A0HE">Musk’s claims about L5</a> last year to be truly jumping the shark.</p><p>Going beyond this attempt at a rational analysis and prediction, since this publication (Pressing Progress) is concerned with promoting technological progress for the benefit of all, and with the potential of autonomous driving so enormous in that regard, it must be our fervent hope that Tesla will lean into its extraordinary tradition of making change happen and find a way to bring full autonomy to its vehicles more quickly than currently seems possible.</p><p><em>By the way</em>, we do see plenty of discussions of them possibly <strong>launching a human driver based service</strong>. This would have fewer technical and regulatory dependencies than teleops, but it would obviously be an enormous undertaking and commitment with a much higher cost structure, and would only make sense as a step towards a hybrid network so the pressure to add the autonomous component of that hybrid would remain. As well, that does not in any way address the need to provide the FSD that Tesla owners believe they have purchased.</p><p>If you have any questions or comments please leave them below. Thanks for reading!</p><h4>Disclosure</h4><p>Nella is a Tesla customer and a (small time) stockholder, but we have no other connection with Tesla. Neither of us are in possession of any inside information pertinent to this post, and we have no association with teleops companies.</p><p>This article expands on a <a href="https://tomdot.medium.com/tesla-fsd-with-teleops-as-fallback-teslops-15e6bf5bafcf">previous one</a> that Tom wrote on the same topic. We must admit that we got a very negative reaction to the idea, but since then we discovered that Voyage had introduced a similar system in 2020, and seeing how similar it was to what we had in mind convinced us to try to make the case again.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=a034a70069f4" width="1" height="1" alt=""><hr><p><a href="https://medium.com/pressing-progress/fsd-now-with-tesla-vision-and-teleops-a034a70069f4">FSD now, with Tesla Vision and Teleops</a> was originally published in <a href="https://medium.com/pressing-progress">Pressing Progress</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Tesla FSD with Teleops as Fallback (Teslops)]]></title>
            <link>https://tomdot.medium.com/tesla-fsd-with-teleops-as-fallback-teslops-15e6bf5bafcf?source=rss-9ca5b6774d6a------2</link>
            <guid isPermaLink="false">https://medium.com/p/15e6bf5bafcf</guid>
            <category><![CDATA[autonomous-vehicles]]></category>
            <category><![CDATA[self-driving-cars]]></category>
            <category><![CDATA[teleoperation]]></category>
            <category><![CDATA[ridesharing]]></category>
            <category><![CDATA[tesla]]></category>
            <dc:creator><![CDATA[Tom Malcolmson]]></dc:creator>
            <pubDate>Sat, 20 Mar 2021 14:02:24 GMT</pubDate>
            <atom:updated>2021-07-13T21:53:50.726Z</atom:updated>
            <content:encoded><![CDATA[<h3>Tesla FSD with Teleops (Teslops)</h3><p>The bull case for the Tesla stock centres around autonomy/FSD: selling it to customers and using it to enable the Tesla ride-hailing network — but there is a consensus that the current state of Tesla’s FSD, the slow pace of improvement, and the continual missing of targets suggest that Tesla is nowhere near <a href="https://www.jdpower.com/cars/shopping-guides/levels-of-autonomous-driving-explained">Level 4</a> and a very long way from creating a ride-hailing service that doesn’t require human drivers.</p><p>The main problem is edge cases. Even once the ‘feature complete’ version of their FSD is working well, the experience of their competitors in this space have shown that that the edge cases are extremely hard to address.</p><p>In deciding whether to buy or sell TSLA I would suggest that this is the most important question: <strong>what is their plan to get from L2/L3 to L4/L5.</strong></p><h3>A Solution — Tesla FSD plus Teleops</h3><p>This problem can be resolved by complementing Tesla FSD with “teleops” (remote control) fallback, where the teleops is used for edge cases with FSD in an enhanced shadow mode.</p><figure><img alt="Women using computer to remotely drive a car." src="https://cdn-images-1.medium.com/max/984/1*oJJ71V8zLHLOrfl9E6O2gg.jpeg" /></figure><p>Actually, I had rather <em>assumed</em> that this was Tesla’s plan: that it is one of the reasons they stuck to their simpler (cheaper) sensor stack while exhibiting such great confidence in the near future potential of their FSD to achieve autonomy. The <a href="https://twitter.com/elonmusk/status/1370522745709158407">recent tweet</a> by Musk about them excluding even radar from the latest incarnation of FSD makes little sense to me otherwise. And if they aren’t planning to utilize teleops (and have no other magic in their back pocket) then I consider <a href="https://www.reuters.com/article/us-tesla-autonomous-idUSKBN24A0HE">Musk’s claims about L5</a> last year to be truly jumping the shark.</p><p>But I’m baffled that I rarely see any reference to this approach. Instead I keep seeing discussion of them starting with a human driver based service, even on <a href="https://electrek.co/2020/02/04/tesla-ride-sharing-app-driver-insurance/">earning calls</a>. Recently I also read/heard Cathie Wood of ARK Investment call on Tesla to launch their ride-hailing service with human drivers, <a href="https://ark-invest.com/articles/analyst-research/tesla-ride-hail-update/">detailed here</a>. This would have fewer technical and regulatory dependencies than teleops, but it still appears to me to be a clearly inferior interim solution — it does not help in providing FSD to Tesla owners — and would be an enormous undertaking and commitment with a much higher cost structure.</p><p>Time for me to try to make the case for Teslops: Tesla FSD augmented with Teleops.</p><h3>Some Details</h3><p>The FSD + teleops solution would be most useful for <strong>location based edge cases</strong>: driving segments that involve difficult situations or uncertainty. Where possible, routing will avoid these (a good idea from a safety perspective anyway) but where that is impossible a remote driver who is experienced in the scenario at hand can be brought online seconds ahead of time. If there is a communications issue then the FSD takes control and behaves as conservatively as possible — maybe even just pulling over to the side of the road.</p><p>For other types of edge cases, eg. <strong>sudden and unexpected events</strong>, teleops could still be used but would be less effective so Tesla would have to focus early efforts on training the initial reaction of their FSD system. I believe that this type of edge case will be much less common then the others (that’s why they are unexpected) and that the FSD can be trained to handle the initial moments of these — before teleops can come online — but this is just my speculation so I consider this one of the weaker aspects of the case I’m making.</p><p>Note that FSD will always be on and available — it will just be in something like shadow mode (the mode currently used when the human driver is in control of the vehicle) when teleops is active but doing additional processing geared mainly towards minimizing the amount of data that needs to be sent. So FSD as fallback to teleops which is fallback to FSD — let’s just say they complement each other very well, and I don’t think teleops makes much sense without some on-board autonomy for driving systems.</p><p>Another weak point in my case is the <strong>availability of the sensors</strong> — it doesn’t seem to me that Tesla has made adequate provision for keeping the cameras clean in adverse conditions. However, I anticipate there being Tesla employees at their charging stations to clean the car while the car is charging, and who would also clean the cameras. I also expect that Tesla will start their network in locations with consistent good weather — perhaps full (unattended) FSD and the ride-hailing network will simply not be available to customers in bad weather? Not ideal for a ride-hailing network but I believe Waymo has had to accept this limitation as well.</p><p>However, if an inability to keep the sensors working properly is a deal breaker for teleops then it is a serious problem for FSD and for Tesla in general, and then why would it not be the current focus of attention and of incremental hardware improvements?</p><p><strong>Regulatory issues</strong> are certainly an important potential impediment, but it is very significant that a fully autonomous commercial service (Waymo) is already operational in the US, and that teleops is not new. As well, since Tesla already has FSD and the hardware that would be used for teleops on most of their vehicles, they already have tons of relevant data and could quickly generate more in support of this new proposition: that their FSD plus teleops should be allowed to operate in full autonomous mode in specific locations. Tesla’s oft declared vision is of FSD everywhere but there is no way of getting around location dependence when we are talking about regulation. And going live everywhere (even just in the US) at launch would be a bad idea for all sorts of practical reasons anway. In the end, if Tesla can demonstrate that this solution is safe then they should be able to achieve regulatory approval in enough states (and the province of Ontario, I hope) to get started, and if they can’t then I am wrong and it shouldn’t be used.</p><p>On a semantic and rather pedantic note, Waymo has made it clear that while their ‘driver’ (their autonomous driving system) may have to rely on human intervention for strategic planning, they are not using teleops to drive the vehicle so their system should be considered fully autonomous. I can well understand that they want to downplay any potential crutches that would allow competitors to close the gap that they have gained with such enormous investment, but people won’t notice or care about this crutch if it works well. Others have focussed on the same distinction, but in the end all that matters is whether the solution is effective in terms of safety and cost.</p><p>I think the primary issue that people will bring up to argue against the use of teleops will be <strong>the reliability and latency</strong> of the cellular communications. I’m not going to get into a detailed, technical discussion of this mainly because it is a quickly moving target:</p><p>* Cellular keeps getting better, in particular, 5G can provide lower latency than 4G and it can allow for <a href="https://spectrum.ieee.org/telecom/wireless/network-slicing-5gs-hottest-feature">reserving slices of spectrum</a> with defined characteristics, at difficult locations.</p><p>* Tesla can continue to reduce the latency of its software stack both in the car and for the operators.</p><p>* The human operators will get better with experience, and potentially more specialized with scale, eg. an operator could specialize in handling roundabouts, flat tires or snowy conditions.</p><p>So that makes the feasibility of this solution a matter of when and where, rather then if, and the best evidence that ‘when’ is soon is that remote controlled robots (as drones, industrial robots, and vehicles) over cellular links are already here — even with on-board autonomy that is inferior to Tesla’s and without 5G network slicing.</p><p><strong>The cost</strong> is another moving target with the number of edge cases that require teleops and the cost of cellular data continually improving. Every use of teleops would be valuable training data that would improve the fallback performance of FSD and eventually eliminate the edge case. Most importantly, my assumption that the cost of this is not a problem is based on the assumption that it will be used for only a very small portion of each trip and on the amount of additional revenue it enables.</p><p>I will throw in one final, enticing consideration … once Tesla vehicles are uploading data and getting back driving instructions to augment their FSD, all in real-time, this opens up new possibilities: the server could resolve some the situations without resorting to human operators, eg. using what it knows from other Tesla vehicles in the area and by applying additional computational resources. This capability would be improved with time and with increasing density of Tesla vehicles, and one would expect that the areas where they offer their ride-hailing service would have lots of Tesla’s, and the places where FSD needed to resort to “outside help” would usually be dense urban areas with high vehicle density. I would imagine that this is a long-term end goal for most of the autonomous driving systems anyway.</p><h3>Prediction</h3><p>This post is more about having a discussion than making hard assertions, but I believe in making concrete predictions where possible, so here is mine: I predict that Tesla will begin employing teleops in their vehicles as a fallback to their FSD before the end of 2022. This will be available to customers who purchased their FSD package and it will be used to enable their ride-hailing service. The locations in which they launch will be determined by weather, cellular, and regulatory considerations.</p><p>I consider that an aggressive timeline for Tesla FSD to improve enough to match the vision I have outlined above, but it is worth considering that, in locations with excellent cellular service, this solution could be launched with a fairly heavy dependence on the teleops as long as there was certainty that this could be improved upon quickly — the extra costs would be manageable since the scale of the service would be quite limited at this point. Waymo is a rather extreme example of starting in a small geographic area!</p><p>I acknowledge, though, that, far from this being the consensus of analysts and pundits etc., the idea of Teslops barely gets mentioned. So I have to ask you what you think about this and what I might be missing?</p><h4>Disclaimer</h4><p>Nella is a Tesla customer and a (very small time) stockholder, but we have no other connection with Tesla. Neither of us have any association with teleops companies.</p><p>This article expands on a previous one that we wrote on the same topic. We must admit that we got quite a negative reaction to the idea back then, but now with the advancing plans of some of the teleops startups lending credence to the viability of our proposal, we felt it was time to press the case again.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=15e6bf5bafcf" width="1" height="1" alt="">]]></content:encoded>
        </item>
    </channel>
</rss>