Introduction to Arti icial Intelligence

Source: Introduction to Artificial Intelligence · Zenodo Authors: Gomaa, Walid Licence: CC-BY-4.0 — https://creativecommons.org/licenses/by/4.0/

W lid Gom Professor t Egypt J p n University of Science nd Technology, Alex ndri, Egypt a a aaaa a a a f

Outline

•What is Intelligence? •The Turing Test. •Short History of AI. •Quantum Computation. •Deep Learning. •Applications of AI. •Challenges. •Demos.

What is Intelligence?

What is Intelligence?

•“The true sign of intelligence is not knowledge but imagination.”, Einstein.

•“I know that I am intelligent, because I know that I know nothing.”, Socrates.

•Three major theories in the 20th centuries.

What is Intelligence?

Charles Spearman [1904]

•Di erent types of intelligence. •Correlated: doing well in some sections of IQ test ➞ tend to do well on all sections. •General intelligence factor g (remains controversial to this day).

ff

What is Intelligence?

Howard Gardner [1983]

•Theory of Multiple Intelligences. •Eight distinct types of intelligence. •Needn’t be correlated. •One can emotionally or socially be intelligent, but not necessarily possess analytical intelligence.

What is Intelligence?

Robert Sternberg [1985]

•Triarchic Theory of Intelligence: •analytic, creative, and practical. •More of a cognitive approach rather than psychometric (scores from intelligence tests) approach.

The Turing Test

The Turing Test

•Aka the imitation game. •Operational de nition introduced by Alan Turing in 1950.

fi

The Turing Test

Short History of AI

[1943-1955] Gestation of AI

• Alan Turing: the Turing test (article ‘Computing Machinery and Intelligence’, 1950).
• McCulloch & Pitts [1943]: model of arti cial neurons (Boolean circuit model of the brain). •Hebb [1949]: simple updating rule for modifying the connection strengths between neurons (Hebbian learning). •Marvin Minsky & Dean Edmonds [early 1950’s]: rst NN called SNARC. fi fi

Short History of AI

1956-The Birth of AI

•Dartmouth meeting to adopt AI (John McCarthy, Marvin Minsky, Claude Shannon, Nathaniel Rochester).

•Conjecture: every aspect of intelligence can be precisely described in a way understandable by the machine. •A. Newell & H. Simon’s Logic Theorist: reasoning system able to prove logical assertions and mathematical theorems.

Short History of AI [1952-1969] Early Enthusiasm & Great Expectations

• General Problem Solver (GPS): imitate human-like planning. • Gelernter's Geometry Engine: proving tricky theorems of geometry.
• Checkers: strong amateur levels. • Lisp: by McCarthy-dominant AI language for 30 years.
• M. Minsky: limited domain problems (calculus integration, blocks world, etc).
• Herbert Simon (1965): “Machines will be capable, within twenty years, of doing any work that a man can do”.

Short History of AI

[1966-1973] More Realistic

•Early machine translation e orts. •Neural network research almost disappears. •H. Simon predicted AI be chess champion and proves a signi cant mathematical theorem within 10 years (it becomes a reality ~40 years).

ff fi

Short History of AI

[1974-1980] The First AI Winter

•US government canceled all funding for machine translation. •Funding for neural networks almost disappeared as M. Minsky and Pappert showed that perceptrons (linear classi ers) are not powerful enough. •1973: Lighthill report ➞ failure of AI, intractability, combinatorial explosion, only toy problems ➞ UK government cut funding for almost all AI research programs.

fi

Short History of AI

[1969-1979] Knowledge-Based Systems

•Domain speci c knowledge in narrow areas of expertise. •Expert systems. •DENDRAL: inferring molecular structure from the information provided by a mass spectrometer.

•MYCIN: diagnose blood infections. Better than junior doctors.

fi

Short History of AI

[1980-present] AI Becomes an Industry

•1982: rst successful expert system RI at the Digital Equipment Corporation

•con gure orders for new computer systems. •By 1986, saving about $40M/year.

•By 1988, DEC AI group had 40 expert systems deployed.

•1981: Japan announced the “Fifth Generation” project, 10 years to build intelligent machine running Prolog. •US responded by forming the Microelectronics and Computer Technology Corporation (MCC) as a research consortium. •UK: Alvey report countered the Lighthill report and AI funding was reinstated.

fifi

Short History of AI

[1986-Present] The Return of NN

•Mid 1950s: the back-propagation algorithm.

•Connectionist models as competitors to symbolic and logical approaches.

Short History of AI

[1987-1990] Second AI Winter

•1987: the collapse of the Lisp machines. •Early 1990’s: expert systems too expensive to maintain, di cult to update, not robust, too specialized.

•Early 1990s: “Fifth generation” project goals have not been met ➞ funding cuts.

•Funding cuts also in the US. ffi

Short History of AI

[1987-Present] Adopting Scienti ic Method

•Opening to other areas: information theory, statistical and stochastic analysis, optimization and control, etc.

•To accept a hypothesis:

•It must be subjected to rigorous experiments. •It must be analyzed statistically for its importance.

•Possibility to replicate experiments and regenerate the results using shared repositories of shared data and code.

f

Short History of AI

[1995-Present] Intelligent Agents

•Internet: widespread applications such as “bots”.

•Integration with other elds such as economics and control theory in order to understand, model, and build intelligent agents (BDI models).

•Looking for a universal algorithms that would learn and act autonomously in any environment.

fi

Short History of AI

[2001-Present] Very Large Data Sets

•Data-oriented rather than algorithmic/rule oriented. •Same algorithm: more data gives better performance. •Learn knowledge from data over hard-coded knowledge.

Short History of AI

[2010-Present] Deep Learning Revolution

•Signi cant advances in machine learning, in particular, deep learning. •Computer vision, speech recognition, and machine translation are dominated by deep learning.

fi

Short History of AI

[Recent] Quantum Computation

•Traditional computing paradigm uses classical physics. •Quantum computation uses quantum physics for its realizations.

Quantum Computation

Quantum Computation

•Qubits replaces classical bits. •Classical bit: either 0 or 1. •Qubit: superposition of 0 and 1: a| 0 > + b| 1 > •Qubit is represented by a wave function.

•Example: the spin of an electron or the polarity of a photon.

Quantum Computation

Essential Properties

• Interference:

• constructive/destructive. • Exploited in quantum algorithms (David Deutsch).

• Entanglement:

• Multiple particles wave functions can become entangled. • Manipulating/measuring one particle necessarily a ects the other.

ff

Quantum Computation

Why Quantum Computation?

• More e cient algorithms.
• Quantum parallelism (machine learning, simulating complex phenomena in physics, chemistry, drug design, developing new materials, etc).
• Breaking classical crypto systems (security/crypt-analysis).
• Building better crypto systems (security/cryptology).
• Simulating quantum systems. ffi

Quantum Computation

Tycoons

•Google Quantum AI research group. •Google Bristlecone: 72-qubit processor ➞ approaching “quantum supremacy”- quantum computers can do calculations beyond the reach of today’s fastest supercomputers. •IBM quantum research: the QISKit (Open Source Quantum Information Software Kit). •IBM 50-qubit processor.

•D-Wave Systems Inc.: quantum computing Canadian company.

Deep Learning

Deep Learning

•Computer vision. •Speech recognition (moving from hidden Markov models). •Natural language processing.

Deep Learning

Deep Learning Paradigm

Traditional programming Data Computer Output Program Machine learning Date Computer Program Output

Deep Learning

Why Now?

•Abundance of data in digitized form.

•Advancements in high-performance computing (GPUs, TPUs, etc).

•Algorithmic advancements, specially training architectures and optimization methods.

Deep Learning

The Philosophy of Neural Networks

•Symbolic AI: intelligence is symbol manipulation by rigid logical rules. •Connectionism: concepts in our brains are not represented by symbols, but by patterns of activations.

Deep Learning

Nonlinear Transformation

: : f(x) : : introducing nonlinearity

Deep Learning MultiLayer Perceptron (Fully Connected NN)

: ⋮ : ⋮ :

input layer hidden layer output layer

Deep Learning

Keyword: Representation

AI

•Representation learning: automatically learning good features or representations. •Deep learning: automatically learn multiple levels of representations of increasing complexity and abstraction.

Deep Learning

Shallow Learning

dog Classi cation

Denoising

OCR “504192”

fi

Deep Learning

Deep Learning

dog ……… Classi cation

Denoising ………

……… OCR “504192”

fi

Deep Learning

Fully Connected Layer

32 × 32 × 3 image ➞ 3,072 × 1 feature vector weight matrix rst layer input x Wx 3,072 × 10 weights 10 1 × 3,072 the dot product of the rst raw of W with x fi fi

Deep Learning

Convolutional Layer

3 activation maps depth corresponds to the 5×5×3 lternumber of lters used 32 28 dim = 75 T w x 28 convolve the lter 32 with the image, slide and compute the dot product new image of size 28 × 28 × 6 using 6 lters fi fi fi fi

Deep Learning Convolutional Neural Network CNN (Convent) Sequence of convolution layers interspersed with activation functions 32 CONV, CONV, 32 CONV, ReLU 28 ReLU 24 ReLU ⋯ 5x5x3 5x5x6 lters lters 28 24 6 10 3 ReLU: f(x) = max{0,x} (gives better performance than logistic functions) introducing nonlinearity (piecewise linear) fifi

Applications of AI

Applications of AI

•Science. •Industry. •Business. •Military.

Applications of AI

Science

•Prediction of protein folding and Development of new drugs (CADD) — AlphaFold 2.

•Telesurgery (remote surgery). •Weather modeling and prediction. •Dermatologist-level classi cation of skin cancer (using deep learning, Nature article). fi

Applications of AI

Industry

•Inspection and sorting systems using machine vision.

•Smart factories and the dawn of Industry 4.0.

Applications of AI

Business

•Advanced nancial systems (predictions of stock market). •Fraud prevention. •Chatbots for customer service. •Risk management.

fi

Applications of AI

Military

•Smart bombs. •Unmanned drone aircrafts. •Robotic soldiers. •Decoding of secret codes.

Challenges

Challenges

•Data bias. •Vast amounts of data. •Wartime robots. •Too specialized. •Transparency problem.

Challenges

Data Bias

•Predictive policing in the US.

Challenges Va s t A m o u n t s o f D a t a

5 •∼ 10 more data required than a human being to understand a concept. •Only big companies (amazon, google, IBM, etc) can a ord that. •Sparsity and scarcity of data in many elds such as healthcare (recognizing tumors in X-ray scans).

Need to develop more ef cient DNN that can work with less data.

fi fi ff

Challenges

Wa r t i m e Ro b o t s

•Establishing responsibility (also applies for autonomous vehicles). •Lowering the threshold for warfare engagement ➞ violating conditions of just war.

•War zone too complex for robots to discriminate targets. •Integration with human ghters.

•Di culties in winning the hearts and minds of the defeated side. •Proliferation of the technology to terrorist groups.

ffi fi

Challenges

Too Specialized

•Very e cient in single task, complete ignorance of any other. •Training for multiple tasks caused interference or forgetting. True AI requires learning multiple tasks (transfer learning, progressive neural networks, Arti cial General Intelligence AGI).

ffi fi

Challenges

Transparency Problem

•NN are are inscrutable to observers. •Why do they work? and make these decisions? •Need to build more coherence into the network from the get-go. Possibly using symbolic logic-based AI.

•Prior knowledge/starting point for understanding the world.

Demos Aggressive Flight (Kumar Lab, UPenn)

Thank You