Forgotten AI Systems: How Syke Was Taught Common Sense

Talk about working with AI is no longer surprising. Only the lazy do not use neural networks, and businesses are increasingly launching their own language models. But few remember that back in the 80s, one ambitious researcher took on the task of developing his own intelligent system with a sense of “common sense.” The project was called Cyc (pronounced as “Sike”), and it still exists – it even has a number of commercial use cases. We are in beeline cloud decided to discuss how it works and what lies at the heart of the solution.

Image by Gabriella Clare Marino - Unsplash.com — Image by Gabriella Clare Marino – Unsplash.com

Origins

Humanity has been interested in AI systems since at least the middle of the last century. Hundreds of scientists, programmers, psychologists and neurophysiologists have sought to develop a computer capable of thinking and independently solving complex problems.

1940. Alan Turing and his colleagues developed a system that successfully “cracked” the Enigma encryption machines during World War II. Many (but certainly not all) consider this episode to be the starting point for the development of AI systems, as well as the scientist’s subsequent experiments, as well as a famous empirical test.

1943. Neurophysiologist Warren McCulloch and a neurolinguist Walter Pitts described the first model of an artificial neuron and suggested that a network of such elements could have high computing power and the ability to self-learn.

1951. Marvin Minsky developed a learning computer based on a randomly connected neural network of forty Hebbian synapses. The system was named SNARCor “stochastic neural-analog calculator with reinforcement.” She decided task of traversing a virtual labyrinth. SNARC selected algorithms using a combined network of potentiometer settings. The latter were physical elements that worked on a principle similar to weights in ML models – they determined the strength of the connections being established.

1958. The next important stage in the development of AI came when the Mark-1 perceptron was born, developed neurophysiologist Frank Rosenblatt. The Mark 1 consisted of four hundred photocells capable of classifying images. In particular, it could recognize letters of the alphabet (including handwritten ones), although the accuracy of identification left much to be desired. Learning occurred through positive or negative feedback and weight adjustments – for these purposes, the Mark 1 were used potentiometers.

1959. They began to look for more practical applications for neural networks. Bernard Widrow And marshian hoff from Stanford developed two AI models based on the same perceptron concept – ADALINE and MADALINE. The first trained recognize binary patterns, so that when analyzing the stream of bits on a telephone line, it could predict each subsequent one. MADALINE, in turn, was used for echo cancellation in telecommunications.

1960s. Developments in the field of AI systems were engaged and domestic specialists. Thus, in the 60s, many scientists were interested in threshold logic, threshold elements, and arithmetic devices based on them. The country's leadership built scientific complexes and research institutes for the development of microelectronics, as well as for conducting neurobiological research. For example, one of such centers was the Research Institute for Physical Problems (NIIPP) in Zelenograd.

At the same time, many specialists were engaged in scientific research and solving theoretical problems in the field of computer science and AI systems. Thus, towards the end of the 60s, D. A. Pospelov developed an approach to decision-making based on the methods semiotic control. Subsequently, it used for managing processes in cargo ports and auto plants.

The author of an ambitious project

At the same time, neural networks of that period were extremely narrowly specialized, and the range of tasks they could solve was extremely limited. Therefore, over time, the enthusiasm around AI systems began to fade. Scientists could not find a new way to overcome the limitations of algorithms, and businesses and governments did not want to fund research without understanding where it could lead.

It was during this difficult time that the future author of one of the most ambitious AI projects, Douglas Lenat, came to the industry.

Who is Douglas Lenat? A researcher and IT specialist, he spent his early years at university focusing on physics and mathematics. His undergraduate thesis focused on creating acoustic holograms, but he later became interested in artificial intelligence. Lenat made a name for himself in the 1970s with a series of intelligent programs. One of these projects was Automated Mathematiciandeveloped at Stanford. AM was one of the first systems that could generate mathematical theorems by modifying Lisp programs (although many of them did not make sense).
Later, the researcher also designed a system EURISKOwhich had no hard-coded heuristics. Lenat used it at a role-playing tournament. game Traveller Adventure 5: Trillion Credit Squadron in the USA. EURISKO built a strategy and Lenat won two national championships.

Douglas Lenat consideredthat it is impossible to give a machine the ability to understand the world without giving it “common sense.” A person does not need to be explained that if you drop a glass of water, it will most likely break and the liquid will leak out. But a computer does not know this by default. Then Lenat decided to collect a massive database, the records in which would describe the logic of the world around us, and provide access to it to the AI system. All that remained was to find money to implement the ambitious idea.

The technological “arms race” and inter-bloc competition helped to make it a reality. The US Department of Defense launched a multi-million dollar program to find promising computer projects. Within its framework, founded MCC, headed by Douglas Lenat and his brother. They began to collect a colossal repository of various knowledge. This is how the Cyc project, or “Syke” (short for the English word encyclopedia), was born.

Under the hood of the “Sike”

To represent human knowledge Cyc uses own language — CycL. When developing it, Lenat's team focused on the syntax of the Lisp family. The knowledge base contains terms (in a sense, these are the building blocks of the Cyc dictionary) and statements. All objects in Cyc are called constants, and logical statements are facts. Objects in the base are divided into collections. With their help, researchers describe the world, as a person sees it, to a computer.

Image by Johnny Briggs - Unsplash.com — Image by Johnny Briggs – Unsplash.com

For example, we know that trees are plants. In “Sike” this information presented as #$Tree-ThePlant. The language also contains the binary predicates #$isa and #$genls. The former asserts that an object is part of some collection, and the latter that the collection itself is a subcollection. For example, the entry (#$genls #$Tree-ThePlant #$Plant) means that all trees belong to the class Plants.

CycL also has Boolean functions. For example, they let you find out whether #$Paris is part of #$France. To write more complex logic, these expressions use the operators #$and, #$or, #$not, and #$implies. In addition to general logic, the researchers added the ability to estimate probabilities to Cyc. The computer must understand that an estimate of a country's population may be approximate, and that the fact that a dog has four legs is true as long as the animal is healthy.

More in the hierarchy of the knowledge base “Sike” present level, called microtheories. These put information into context, as engineers sought to build a universal database covering dozens of scientific and practical areas. For example, a microtheory might be generated that integrates a particular person’s knowledge and beliefs so that the system can approach a problem or question from that person’s perspective. Another microtheory might recognize that Newtonian and quantum physics don’t always agree, but allow the machine to apply one of the two models depending on the situation.

All this allows Cyc to make logical inferences. For example, if you feed the system the statement that a certain person went and bought himself some sweets at a certain time, Cyc will assume [PDF, стр. 43]whether the purchase was paid for in cash, whether the person planned it in advance, how much money he had with him, etc. To do this, the system calculates probabilities and is guided by the information loaded into the database about the time period, the economic situation, the range of establishments, etc.

Another component of the Sike project is the development tools and interface mechanisms that allow view and extract data from the knowledge base. This can then be used to query a search engine or passed to other parts of a large-scale database management system.

A difficult fate

By 1989, the Cyc database contained a million different expressions, including 50,000 individual objects and 6,000 collections. Most of the time, engineers manually entered data into the system. But over time, researchers added the ability to download data from Wikipedia.

To date, more than 25 million rules have been added to the Cyc root database. In total, the entire team spent about 2 thousand man-years on its implementation. But the future of the project seems more uncertain than ever. The last time major news about it came back in 2016, when Lenat statedthat Cyc began to be used in one of the hospitals in American Cleveland to automate the selection of patients for clinical trials. Cyc was also used in the financial sector. The developers said that their system helped to identify a case of insider trading.

Unfortunately, last year the main author of the idea and the head of the project died at the age of 72. Not long before that, Lenat managed to release a separate article about how he sees the future of Cyc in conjunction with machine learning and LLM, and why the latter cannot be considered real AI. Next time, we will talk about what prospects Cyc has today and how it has influenced the development of specialized startups.