Jeff Dean recently discussed the beginnings of Gemini, detailing its design principles. Initially, Gemini faced some delays due to its coding capabilities, but enhancing these capabilities ultimately boosted the model’s reasoning skills.
The beginning of Gemini
Dean discovered Gemini when he noticed several teams working independently towards a common objective, which he found to be a peculiar strategy.
Dawn Song, the interviewer, inquired:
What aspects of Gemini’s development caught you off guard, and what insights from creating Gemini do you believe can influence the future generation of AI systems?
Dean responded.
Gemini represents the integration of various previous research projects conducted at Legacy DeepMind, Google Brain, and other divisions of Google Research.
We all discovered that we were moving in similar directions, focusing on training and increasing the size of models. Additionally, we were working independently on making language models capable of understanding images and other multimodal data.
I composed a brief memo suggesting that instead of working separately, we should collaborate by combining our teams, ideas, computing resources, and expertise to create a single multimodal model involving the top talents from various research entities at Google.
I believe that was quite positive.
We found a way to work together more closely, which has been very successful.
Jeff Dean’s screenshot
Gemini is considered multimodal due to several reasons.
Gemini was intentionally designed to be able to comprehend various forms of information, including audio, images, language, and text. An unexpected advancement in its reasoning occurred when it was trained for coding.
Jeff Dean provided an explanation.
The emphasis on ensuring the model is multimodal from the start has been highly beneficial. It’s essential for the model to comprehend various modalities like text, language, code, images, videos, audio, and more for versatile usage.
We include some LiDAR data in the training data to ensure that the model recognizes its importance for future Gemini model training.
I believe it was a successful decision from the start to incorporate that feature into all current Gemini models. Our goal was to ensure the model excelled in various capabilities.
I believe our emphasis on excelling in coding was lacking, so we made efforts to improve in that area.
I believe we are making progress with our current initiatives. By concentrating on this, we can develop a system capable of effectively analyzing complex problems by breaking them down into smaller components.
Improving coding skills can also enhance abilities in non-coding tasks.
Insight on Education
It is intriguing to see how expertise in one area can bring advantages in other areas, similar to how humans become experts in various fields. Long ago, a samurai sword master named Miyamoto Musashi emphasized the importance of learning different arts and professions beyond martial arts. For instance, he used carpentry as an illustration to demonstrate how mastering carpentry skills can be applied to sword fighting by understanding tools, planning, managing workers, and comprehending building structures.
Jeff Dean’s explanation of Gemini’s inception is valuable as it enhances users’ comprehension of Gemini, highlighting the potential of viewing it as a multimodal model rather than solely a generative text model.
Jeff Dean’s Key to Innovation and Achieving Success
Dawn Song inquired:
One notable aspect of your career is that many of the ideas you have worked on, such as MapReduce and MOE, took years before their significance was recognized by the wider community.
You possessed the ability to turn everything you touched into gold and the unique ability to create and enhance ideas and systems.
Many audience members may be interested in learning how you achieve it.
How can you differentiate between technologies that are truly fundamental and those that are just trendy?
What engineering principle has endured over time in your career?
What new concepts will be necessary in the coming years and decades as AI transitions from models to more independent agents?
Jeff Dean replied:
I believe I have been fortunate, which may have played a role. Additionally, I make an effort to monitor various trends and research areas being investigated by the community.
I frequently advise students to skim through multiple papers rather than reading one in depth. By doing so, they can gather multiple ideas and possibilities to expand their knowledge.
You aim to link significant ideas that have not been linked before.
Sometimes pondering a challenging issue becomes easier when you consider potential solutions that are beginning to take shape in your mind. This can assist in forming a comprehensive solution to a problem, making it appear less daunting by identifying key aspects that show promise.
There are two extra tasks that I am unsure how to accomplish, but with effort, I believe I can figure them out.
Sometimes it is crucial to make small improvements to our current processes, but it is also essential to explore alternative approaches that could be feasible in the near future.
I have also attempted various things that did not yield successful results, which is a valuable piece of advice. Experiment with different options, even if they may not succeed. Some of them will.
Watch Jeff Dean’s Interview.
Image displayed during interview