Architecture Theory & Design

77 beliefs (77 IN, 0 OUT)

• IN a3c-mnih-2016-async-actor-critic
  A3C (Asynchronous Advantage Actor-Critic) was introduced by Mnih et al. (2016) using multiple parallel agents updating a shared model
• IN actor-critic-combines-value-and-policy
  Actor-critic methods combine value function estimation (critic) with direct policy optimization (actor) and belong to the Generalized Policy Iteration (GPI) class
• IN alphago-two-cnn-networks
  AlphaGo used two CNNs — a policy network for move selection and a value network for position evaluation — driving Monte Carlo Tree Search
• IN architecture-selection-two-stage-filter
  ML architecture selection operates as a two-stage filter: data geometry determines which inductive biases are effective (first filter), and hardware scalability determines which effective architectures survive (second filter) — scalability can veto geometric fit but not vice versa, explaining why theoretically superior architectures are routinely displaced.
• IN autonomous-vehicles-use-slam
  Autonomous vehicles use computer vision for navigation, SLAM (Simultaneous Localization and Mapping), obstacle detection, and event detection.
• IN cv-and-cg-are-inverses
  Computer vision and computer graphics are inverses: CV extracts 3D models from 2D images, while CG renders 2D images from 3D models; augmented reality combines both.
• IN cv-began-1960s-mit-summer-project
  Computer vision originated in the late 1960s at AI-pioneering universities; the 1966 MIT summer project famously underestimated the difficulty of describing what a camera sees.
• IN cv-field-evolution-three-eras
  Computer vision evolved from digital image processing (1980s) to geometric/mathematical vision (1990s) to algorithmic/learning-based vision (2000s-present)
• IN cv-market-size-2024
  The three largest computer vision market sectors in 2024 by size are: Industry ($5.22B), Medicine ($2.6B), and Military ($996.2M).
• IN cv-system-pipeline-six-stages
  A computer vision system pipeline has 6 stages: image acquisition, pre-processing, feature extraction, detection/segmentation, high-level processing, and decision making
• IN cv-three-recognition-types
  Computer vision recognition has three types: object recognition (class), identification (instance), and detection (location)
• IN cv-vs-machine-vision-distinction
  Computer vision is the broader scientific research field while machine vision refers specifically to industrial/manufacturing application focus
• IN decision-transformer-rl-as-sequence-modeling
  Decision Transformer (Chen et al., 2021) casts reinforcement learning as sequence modeling using the Transformer architecture
• IN deep-tamer-2018-arl-ut-austin
  Deep TAMER (2018, developed by ARL and UT Austin) combined deep learning with human evaluative reinforcement feedback for robot task learning from video or in-person demonstration.
• IN dqn-cnn-plus-qlearning
  Deep Q-Networks (DQN) combine CNNs with Q-learning for reinforcement learning directly from high-dimensional sensory input, demonstrated on Atari 2600 games in 2015
• IN dqn-discrete-actions-continuous-states
  DQN bridges discrete action spaces with continuous state spaces; DDPG extends this to continuous action spaces
• IN dqn-mnih-2015-human-level-atari
  Mnih et al. (2015) extended Q-learning to Deep Q-Networks (DQN), achieving human-level performance on Atari games
• IN dyna-algorithm-learns-model-generates-simulated-transitions
  The Dyna algorithm learns a model from real experience and generates simulated transitions to supplement real ones for value function updates, improving sample efficiency
• IN dyna-architecture-sutton-1990
  The Dyna architecture integrating learning, planning, and acting in model-based RL was introduced by Sutton (1990)
• IN effective-architectures-are-geometry-matched-compression
  Since prediction and compression are formally equivalent, and data geometry (the manifold hypothesis) offers a geometric explanation for why certain architectural inductive biases succeed, effective ML architectures can be understood as implicit compression algorithms whose success depends on alignment with the data's intrinsic geometry. CNNs exploit spatial locality and transformers exploit relational structure, and this framework suggests they succeed when the data's geometric properties match their compression strategy — though the formal connection between manifold geometry and the prediction-compression equivalence remains conceptual rather than proven.
• IN egomotion-vs-optical-flow-vs-tracking
  Egomotion is the camera's own 3D rigid motion, optical flow is per-pixel apparent motion, and tracking is following specific objects across frames
• IN epsilon-greedy-exploration-strategy
  The ε-greedy strategy selects the best-known action with probability (1 − ε) and a random action with probability ε to balance exploration and exploitation
• IN experience-replay-lin-1992
  Experience replay was introduced by Lin (1992) as a technique for self-improving agents, and later became a key component of DQN
• IN feature-engineering-is-preprocessing-before-training
  Feature engineering is a preprocessing step that occurs before model training, transforming raw data into more effective model inputs.
• IN feature-explosion-too-many-features-solved-by-regularization
  Feature explosion occurs when identified features become too numerous for effective model estimation; it is addressed by regularization, kernel methods, or feature selection.
• IN feature-store-centralized-repo-training-and-inference
  A feature store is a centralized repository for storing, organizing, versioning, and serving features for both model training and production inference.
• IN gan-fid-lower-better-is-higher-better
  FID (Fréchet Inception Distance) measures distance between generated and real distributions (lower is better); IS (Inception Score) measures output diversity and quality (higher is better) — both use Inception-v3
• IN generative-vs-discriminative-models
  Generative models (naive Bayes, LDA) model the joint distribution P(x,y) and can have closed-form solutions; discriminative models (logistic regression) model P(y|x) directly and generally require iterative optimization
• IN graphcast-10-day-weather-under-one-minute
  GraphCast predicts weather up to 10 days globally in under a minute, matching state-of-the-art traditional PDE-based systems that take hours
• IN hartley-zisserman-multiple-view-geometry
  Hartley & Zisserman's 'Multiple View Geometry in Computer Vision' (2003) is the definitive reference for epipolar geometry, fundamental matrices, and 3D reconstruction from images
• IN image-processing-vs-cv-input-output
  Image processing takes images as input and produces images as output (2D transformations), while computer vision takes images as input and produces interpretations, content analysis, or 3D models as output.
• IN inverse-rl-infers-reward-from-demonstrations
  Inverse reinforcement learning (IRL) infers the reward function from observed expert behavior; MaxEnt IRL is a special case of Random Utility IRL (RU-IRL)
• IN inverse-rl-ng-russell-2000
  Inverse reinforcement learning was defined by Ng & Russell (2000) as the problem of recovering reward functions from observed behavior
• IN ius-three-abstraction-levels
  Image-understanding systems operate at three abstraction levels: low (edges/textures/regions), intermediate (boundaries/surfaces/volumes), and high (objects/scenes/events)
• IN latent-diffusion-model-behind-modern-image-generation
  The latent diffusion model is the specific architecture behind modern image and video generation systems, not raw pixel-space diffusion.
• IN machine-vision-is-industrial-cv
  Machine vision is the application of computer vision specifically in industrial settings (factory automation, inspection, process control), emphasizing real-time processing, controlled conditions, and integration with control theory.
• IN manifold-geometry-non-biological-architecture-foundation
  The manifold hypothesis offers a non-biological theoretical lens for understanding inductive bias effectiveness — if high-dimensional data lies on low-dimensional manifolds, then architectures exploiting local connectivity and weight sharing can be understood as responses to data geometry rather than ad hoc engineering choices or neuroscience analogy, suggesting that architectural effectiveness may track manifold geometry matching rather than biological fidelity.
• IN manifold-structure-explains-inductive-bias-effectiveness
  The manifold hypothesis offers a theoretical lens for understanding why certain architectural inductive biases are effective — if high-dimensional data lies on low-dimensional manifolds (and prediction is formally equivalent to compression of that structure), then architectures exploiting local connectivity and weight sharing can be seen as responses to data geometry rather than purely ad hoc engineering choices.
• IN marr-vision-1982-foundational
  David Marr's 'Vision' (1982) established the computational theory of vision and is considered the foundational theoretical work for the field
• IN max-entropy-irl-ziebart-2008
  Ziebart et al. (2008) introduced maximum entropy inverse reinforcement learning
• IN mediapipe-google-cv-edge-framework
  MediaPipe is Google's open-source framework for deploying computer vision models to edge devices, supporting face detection, image classification, and object detection.
• IN mit-summer-vision-project-1966
  The MIT Summer Vision Project (Seymour Papert, 1966) proposed solving computer vision 'in a summer,' famously underestimating the difficulty of the problem
• IN ml-bayesian-networks-dags
  Bayesian networks are probabilistic graphical models using directed acyclic graphs (DAGs) to represent conditional independence among random variables; dynamic Bayesian networks handle sequential data
• IN ml-breiman-two-cultures-2001
  Breiman's 'Statistical Modeling: The Two Cultures' (2001) is a landmark paper on the divide between data-modeling (statistics) and algorithmic-modeling (ML) approaches
• IN ml-logistic-regression-classification
  Logistic regression is used for statistical classification despite having 'regression' in the name
• IN ml-manifold-hypothesis
  The manifold hypothesis proposes that high-dimensional data lies along low-dimensional manifolds, and is a foundational assumption for many dimensionality reduction techniques
• IN ml-mitchell-1997-formal-definition
  Tom Mitchell's (1997) formal definition: a computer program learns from experience E with respect to task T and performance measure P if its performance on T as measured by P improves with experience E
• IN ml-pattern-recognition-same-field
  Machine learning and pattern recognition are described as 'two facets of the same field' (Bishop, 2006)
• IN ml-pca-most-popular-dimensionality-reduction
  Principal Component Analysis (PCA) is the most popular dimensionality reduction method, projecting higher-dimensional data to lower-dimensional space by extracting principal variables
• IN ml-rl-environments-modelled-as-mdps
  Reinforcement learning environments are typically modelled as Markov Decision Processes (MDPs), and RL algorithms do not require exact mathematical models of the MDP
• IN ml-st-georges-1988-bias
  St. George's Medical School (1988) is an early documented case of algorithmic discrimination where a computer program denied approximately 60 candidates based on gender or non-European names
• IN ml-vs-datamining-distinction
  Machine learning focuses on prediction from known properties while data mining focuses on discovery of previously unknown properties; they share methods but differ in goals
• IN monte-carlo-rl-requires-episodic-tasks
  Monte Carlo RL methods require episodic tasks and update only after episode completion; they do not require knowledge of environment dynamics
• IN nmf-nonnegative-yields-part-based-representations
  Non-Negative Matrix Factorization (NMF) with non-negativity constraints yields part-based representations with natural clustering properties.
• IN offline-rl-uses-historical-logs-no-live-interaction
  Offline RL optimizes policies using fixed datasets of past interactions (e.g., conversation logs) without requiring live environment interaction during training
• IN optimal-policy-deterministic-stationary
  An optimal RL policy can always be found among deterministic stationary policies — no need to search stochastic or history-dependent policies
• IN pca-ica-unsupervised-lda-supervised-dimensionality-reduction
  PCA and ICA are unsupervised dimensionality reduction techniques, while LDA (Linear Discriminant Analysis) is supervised.
• IN pca-linear-special-case-of-manifold-hypothesis
  PCA can be understood as a linear special case of the manifold hypothesis — it assumes data lies near a linear submanifold and finds an optimal projection onto it. This connection may partly explain PCA's popularity as a dimensionality reduction method: it represents the most tractable (and most restrictive) instance of manifold-based thinking about data representation.
• IN pomdp-extends-mdp-partial-observability
  When the agent observes only a subset or noisy observations of the state, the problem becomes a Partially Observable MDP (POMDP) rather than a standard MDP
• IN ppo-trpo-a3c-on-policy-advantage
  PPO, TRPO, and A3C are on-policy algorithms that use the advantage function, while SAC, TD3, and DDPG are off-policy algorithms
• IN prediction-compression-manifold-unified-view
  Prediction and compression are formally equivalent (Delétang et al., 2023), and the manifold hypothesis — that high-dimensional data lies on low-dimensional manifolds — offers a geometric explanation for why compression is effective in practice. Together, these ideas suggest a connection between learning, compression, and geometry, though the formal link between the manifold hypothesis and the prediction-compression equivalence is conceptual rather than proven.
• IN q-learning-originated-watkins-1989
  Q-learning was introduced in Chris Watkins' 1989 PhD thesis
• IN q-star-sufficient-for-optimal-action
  Knowing Q* (the optimal action-value function) alone is sufficient to act optimally by choosing argmax_a Q*(s,a), whereas V* alone requires the transition model
• IN regret-bounds-auer-jaksch-ortner-2010
  Auer, Jaksch & Ortner (2010) established near-optimal regret bounds for reinforcement learning, a key theoretical result for exploration-exploitation tradeoffs
• IN reinforce-williams-1987
  The REINFORCE policy gradient algorithm was introduced by Williams in 1987
• IN reinforcement-learning-modeled-as-mdp
  Reinforcement learning is formally modeled as a Markov decision process (MDP) with state transitions governed by probability distributions.
• IN rl-called-approximate-dynamic-programming
  Reinforcement learning is also known as 'approximate dynamic programming' or 'neuro-dynamic programming' in operations research and control theory literature
• IN rl-environment-modeled-as-mdp
  The RL environment is typically modeled as a Markov decision process (MDP) with four components: state space S, action space A, transition probability P, and reward function R
• IN rl-is-third-ml-paradigm
  Reinforcement learning is one of three core machine learning paradigms alongside supervised and unsupervised learning, distinguished by learning from interaction and reward signals rather than static datasets
• IN rlhf-and-self-supervised-categorized-core-ai
  RLHF (Reinforcement Learning from Human Feedback) and self-supervised learning are categorized as core AI concepts, not techniques specific to LLMs.
• IN rlhf-connects-base-lm-to-aligned-chatbots
  RLHF (Reinforcement Learning from Human Feedback) is the key technique connecting base language models to aligned, deployable chatbots.
• IN rlhf-trains-reward-model-from-human-preferences
  RLHF trains a reward model from human preference ratings, then uses that reward model to guide RL policy optimization — it is not direct human-in-the-loop RL
• IN sutton-barto-canonical-rl-textbook
  Sutton & Barto 'Reinforcement Learning: An Introduction' (2nd ed., 2018) is the canonical RL textbook, freely available at incompleteideas.net
• IN svm-rbf-kernel-infinite-dimensional-space
  The RBF (Gaussian) kernel can map data to an infinite-dimensional feature space.
• IN td-learning-lineage-sutton-watkins-mnih
  Temporal-difference learning has a direct lineage: Sutton (1984) → Watkins Q-learning (1989) → Mnih et al. DQN (2015)
• IN td-learning-sutton-1988
  Temporal-difference learning was formally introduced by Sutton in 1988, predating deep RL by decades
• IN vpus-emerged-circa-2016
  Vision Processing Units (VPUs) emerged circa 2016 as specialized processors for computer vision workloads alongside CPUs and GPUs