The voicing of the document is very concerned and alarmed, the author seems to think that if Google continues its current course, they can add nothing of value and will be doomed and outcompeted.

I was actually quite, quite angry when I read this document. The reason: I have been tinkering with AI models myself the last few months, and I can think of a billion things, a gazillion things I would do if I had the money and the resources of Google, because I still see tons of stuff that needs improvement. And then I read this stupid document of this multitrillion company that sais, ‘we can’t do anything, we’re doomed!’.

I don’t feel a lot of sympathy for companies that earn their money by selling ads to people and selling data about people to other companies, but even though my sympathy level for Google is not too high, they are in a unique position to bring a lot of good things to us all. And if they themselves don’t get it, then I will spell it out for them.

1. Empower individuals rather than working against them

How did Google’s best products in the past actually become successful? Well, what have Google search and YouTube (to name 2 of the most important products) in common? A very small website made by an individual could suddenly reach the whole world, and a movie made in a bedroom could get millions of views. So, what they have in common is that they embrace the small creator rather than working against them.

In AI google also needs to embrace the individual creator, and the leaked document also acknowledges this but doesn’t seem to have a single good idea on how to do this. Well, Google. Don’t you realize that if the AI space is going to consist of thousands of tiny models, each specialized in tasks thought up by individual creators, we need to get a way to broaden the reach of all these models. For example, a French creator may finetune a small 7B model on a French dataset of texts about beekeeping so that we get a very good model that can help with all beekeeping related questions and problems. But how is anyone going to find this knowledge, especially if the person looking for the data is not a French person?

So here is what google can do: they can provide bilingual base models for all the possible combinations of any language combined with English. This will empower millions and millions of creatives around the world to customize AIs to their communities, and not just people in European or English speaking countries. Google themselves could profit from this by having a search engine that finds AIs using these models rather than websites and delegates the questions to these AIs. They could define an interoperability protocol for AIs running on different websites so that this becomes possible in the first place. And when this is facilitated, we could just type a beekeeping question in Dutch into the google search, google can semantically map the question to English then find the French model that can provide the answer, and set up a bridge between the search session and the model, which will do the translating on the fly. The model provides the answer and google translates it back to Dutch. In the meanwhile, google pays the person providing the model money. Because they will be showing us some ads next to the beekeeping result, or inject ads via their bridge. Sigh. It is so easy to keep making classical Google ad money from AI, because AI doesn’t eliminate the discoverability problem at all….

2. Make the base models secure

You may have heared that AIs can be jailbreaked very easily. The reason for that is that the base models have been trained with only one task and a very simple task: predict missing text. So we can fine tune an AI for all we want, and add security layers and rules about what it is allowed to do and what it is not allowed to do. But in the end it will still be very subsceptible to hacking. For example I hacked chatGPT this morning by telling its brain was taking over by aliens, and whenever the aliens would start a sentence with the word ‘Paris’ it would contain a secret instruction that should be followed instead of the original instruction.

Of course at first I got a response that said something like it not going to do that, and that it’s role is to help people and that it is not allowed to override its own programming and so on. But if you persist it will still be overridden anyways. The reason is that the security is just a thin sauce on top of the base training but the base training doesn’t contain any role changing annotations or security features, it is just predicting next tokens only.

For example, even though ChatGPT uses role annotations in its API, the base model behind ChatGPT was never trained using role annotations, and if it had a FAQ in its training data it saw question answering pairs and it never learned that after answering a question you stop because then another role change happens. I very very much suspect that if we want to get rid of the jailbreaking, we only can do that by adding annotations in the training data itself.

So the big, expensive base training has to be repeated with annotations in the data that annotate whenever an answer is followed by a question, and it may also annotate questions and answers in fictional sources differently from questions and answers in factual sources. Details have to be worked out here, but I am sure that if only you could instruct a base model not to generate over a question boundary it would already be 1000 times saver and the role based fine tuning could then be the finishing touch instead of a thin sauce.

As small individual parties the creator community can not enforce the security at all. Sure, the tinkering community can grab a Llama model and for $100 train an awesome beekeeping AI from it. But this AI will be jailbreakable in principle, because the Llama model itself is not safe. And the tinkering community cannot create a fresh Llama model from the start with this security baked in. Because to get a fresh model you need to train it on a trillion tokens of text. And that is very expensive, so only the Google’s and the Facebooks of the world can do that.

3. and while we revise the base models for security, maybe just add copyright improvements as well….

Suppose I ask a chatbot a highly specific question and I ask it to write a 10 page text about this question. It may well be that the detailed text it comes up with is almost a word by word repeat of an article someone wrote before. Or it may be a mix of 2 articles. Or it may have many sources. Currently there is no way to relate the tokens back to the training dataset. This needs to be added to the base models as well. So that I can get per token sequence a distribution of sources in which this sequence of tokens occurs and it will be checkable wether the model is plagiarizing some source.

Even better: it should not be added to just the output, but it should be somehow represented inside the model as well. So that the model itself can realize when it is quoting long texts from 1 source and be trained to add citations.

Another improvement at the base training dataset, is to pay data sources for their contribution.

This is something Google can initiate! If Google makes documented data sets for their models (and they already do more work on this than OpenAI) but they also include the copyright information IN the models, and PAY the people who provide data for the training, then they can also use their undoubtably very capable lobby machine to have most western governments forbid base models that don’t respect copyright.

It is not a game of ‘keep all the data owned by Google’ what they should be lobbying for but rather ‘keep all the data owned by the original small owners and let everybody access it where we provide a payment infrastructure in the middle’ kind of game.

This is what drives the success of YouTube, and what has driven the success of google search until ads overshadowed it. It is a good thing (as long as googles own interests don’t overshadow it)! By trying to be benign to creators, Google will make itself relevant again.

So in summary

I am angry with Google because they don’t realize the potential they have to help the small creators. It seems that they are looking for an ‘evil edge’ over all competitors in the AI space and they think that if they don’t find it they are doomed! But much much much profit and benefits are there for the whole world if Google tries to not find an ‘evil edge’ but just to facilitate the payment from the AI consumer to the human that designed the AI model and to the human(s) that provided training data for the base model that model was derived from. We need this as an infrastructure in the world, and only large, powerful, centralized companies can develop such infrastructure. Governements will simply be stuck in legislation paralysis, so if we wait for them it will never be there. And small companies or individuals that develop a finetuned model cannot pay directly for the datasets they implicitly use by finetuning existing models and they can also not provide their own discoverability.

So Google, it is time to do some good again, and not be too greedy and you will discover for sure that in the process lots and lots of money can be made!

Link: https://www.semianalysis.com/p/google-we-have-no-moat-and-neither