by “The PaLM 2 language model is stronger in logic and reasoning, thanks to extensive training in logic and reasoning,” Google CEO Sundar Pichai said at the company’s I/O conference. “He is also trained on multilingual texts covering more than 100 languages.”
PaLM 2 is much better for a number of text-based tasks, Google’s senior research director Slav Petrov told reporters. “It is significantly improved compared to PaLM 1 [το οποίο ανακοινώθηκε τον Απρίλιο του 2022]Petrov said.
As an example of its multilingual capabilities, Petrov showed how PaLM 2 is able to understand idioms in different languages, giving the example of the German phrase “Ich verstehe nur Bahnhof”, which literally translates to “I only understand the station”, but is better understood as “I don’t understand what you are saying” or, as an English idiom, “everything is Greek to me”.
In a research paper describing the capabilities of PaLM 2, Google engineers claimed that the system’s language proficiency is “sufficient to teach that language” and noted that this was partly due to the higher prevalence of texts no English in training data.
Like other large language models, the creation of which requires enormous amounts of data, time and resources, PaLM 2 is not so much a single product as a family of products. Its different versions should be deployed in consumer and professional environments. The system is available in four sizes, named Gecko, Otter, Bison, and Unicorn, from smallest to largest, and is configured with domain-specific data to perform certain tasks for enterprise customers.
Think of these customizations as taking a basic truck chassis and adding a new engine or front bumper to allow it to perform certain tasks or perform better in certain terrains. There is already a version of PaLM trained on health data (Med-PaLM 2), which Google says can answer questions similar to those on US medical licensing exams at the “specialist” level. Another trained in cybersecurity data (Sec-PaLM 2), which can “explain the behavior of potential malicious scripts and help identify threats in code,” Petrov said. Both of these models will be available through Google Cloud, initially for select customers.
As for Google itself, PaLM 2 is already behind 25 company features and services, including Bard, the company’s experimental chatbot. Updates available through Bard include improved coding capabilities and better language support. It is also used for artificial intelligence functions in Google Workspace online applications such as Docs, Slides and Sheets.
In particular, Google claims that the lite version of PaLM 2, Gecko, is small enough to run on mobile phones, processing 20 tokens per second – the equivalent of around 16 or 17 words. Google didn’t specify what hardware was used to test this model, other than that it works “on the latest smartphones.” Nevertheless, the miniaturization of such language models is important. Such systems are expensive to run in the cloud, and being able to use them locally would have other benefits, such as improved privacy. The problem, of course, is that smaller versions of language models inevitably perform less well than their older siblings.
With PaLM 2, Google hopes to bridge the “artificial intelligence gap” between the company and rivals such as Microsoft, which has aggressively pushed artificial intelligence language tools into its Office software suite. Microsoft now offers artificial intelligence features that can summarize documents, compose emails, create slides for presentations, and more. Google will have to introduce at least equal functionality or risk being seen as slow to implement its AI research.
While PaLM 2 is certainly a step forward for Google’s work on AI language models, it suffers from issues and challenges common to the technology at large.
For example, some experts are beginning to question the legitimacy of the training data used to build linguistic models. This data usually comes from the Internet and often includes copyrighted texts and pirated e-books. The tech companies building these models refuse to answer questions about where their training data comes from. Google continued this tradition in its description of PaLM 2, noting only that the central training part of the system consists of “a diverse set of sources: web documents, books, code, mathematics and chat data”, without giving further details. .
There are also the well-known problems in the results of AI language models such as “hallucinations” or the tendency of these systems to simply make up information. Speaking to The Verge, Google’s VP of Research, Zoubin Ghahramani, said that in this respect, PaLM 2 is an improvement over previous models “in the sense that we are making a huge effort to improve continuously the indicators of performance and good fortune”. At the same time, he notes that the industry as a whole “still has some way to go” in combating fake news generated by artificial intelligence.
