AI Glossary

Supervised learning is a machine learning approach where models learn from labeled training data, mapping inputs to known correct outputs.

Unsupervised learning is a machine learning approach where models find patterns and structures in data without labeled examples or predefined outputs.

Semi-supervised learning combines a small amount of labeled data with a large amount of unlabeled data to improve model performance beyond what either could achieve alone.

Reinforcement learning trains AI agents to make sequential decisions by rewarding desired behaviors and penalizing undesired ones through interaction with an environment.

One-shot learning enables models to learn new concepts from a single example, commonly used in face recognition and image classification tasks.

Active learning is a strategy where the model selects which data points should be labeled next, focusing human annotation effort on the most informative examples.

Federated learning trains AI models across multiple devices or organizations without sharing raw data, preserving privacy while enabling collaborative model improvement.

Curriculum learning trains models by presenting training examples in a meaningful order, typically from easy to hard, mimicking how humans learn progressively.

Online learning updates the model incrementally as each new data point arrives, rather than training on the entire dataset at once.

Batch learning trains models on the entire dataset at once, as opposed to online learning which processes examples incrementally.

Domain adaptation transfers a model trained on one data distribution (source domain) to work effectively on a different but related distribution (target domain).

Classification is a supervised learning task where the model predicts which category or class an input belongs to, such as spam detection or image recognition.

Regression is a supervised learning task where the model predicts a continuous numerical value, such as price, temperature, or probability.

Clustering is an unsupervised learning task that groups similar data points together without predefined labels, discovering natural structures in data.

Time series forecasting predicts future values based on historical temporal data patterns, used for demand planning, financial analysis, and resource allocation.

Random forest is an ensemble method that combines predictions from many decision trees trained on random subsets of data and features for more accurate, robust predictions.

Gradient boosting builds an ensemble of decision trees sequentially, where each new tree corrects the errors of the previous ones, achieving state-of-the-art results on tabular data.

A support vector machine finds the optimal hyperplane that separates classes with the maximum margin, effective for high-dimensional classification tasks.

K-nearest neighbors classifies data points based on the majority class among their k closest neighbors in feature space, a simple but effective non-parametric method.

Naive Bayes is a probabilistic classifier that applies Bayes theorem with a naive independence assumption between features, effective for text classification.

K-means is a clustering algorithm that partitions data into k groups by iteratively assigning points to the nearest centroid and updating centroids.

DBSCAN is a density-based clustering algorithm that groups together closely packed points and identifies outliers as points in low-density regions.

A Gaussian mixture model represents data as a combination of multiple Gaussian distributions, providing probabilistic soft clustering with cluster membership probabilities.

A hidden Markov model is a probabilistic model for sequential data where the system transitions between hidden states that generate observable outputs.

A Bayesian network is a probabilistic graphical model that represents variables and their conditional dependencies as a directed acyclic graph.

A genetic algorithm is an optimization method inspired by natural evolution that evolves a population of solutions through selection, crossover, and mutation.

AdaBoost is an ensemble method that combines multiple weak classifiers by weighting them based on their accuracy and focusing on hard-to-classify examples.

Isolation forest is an anomaly detection algorithm that identifies outliers as data points that are easy to isolate through random partitioning.

Expectation Maximization is an iterative algorithm for finding maximum likelihood parameters in models with latent variables, used to train Gaussian mixture models and HMMs.

Autoencoders are neural networks that learn compressed data representations by training to reconstruct their inputs, used for dimensionality reduction and anomaly detection.

The training set is the portion of data used to train a machine learning model, from which the model learns patterns and relationships.

The validation set is data held out during training to tune hyperparameters and monitor for overfitting, guiding model selection decisions.

The test set is data held out completely during training and validation, used only once for final unbiased evaluation of model performance.

Cross-validation is a model evaluation technique that partitions data into multiple folds, training and testing on different splits to get a robust performance estimate.

Class imbalance occurs when training data has significantly more examples of some classes than others, causing models to be biased toward the majority class.

Oversampling increases the number of minority class examples in a training set by duplicating or generating synthetic examples to address class imbalance.

SMOTE (Synthetic Minority Over-sampling Technique) creates synthetic training examples for the minority class by interpolating between existing minority samples.

Data preprocessing transforms raw data into a clean, structured format suitable for machine learning, including cleaning, normalization, and feature engineering.

Feature engineering creates new input variables from raw data to improve model performance, leveraging domain knowledge to extract predictive signals.

Feature selection identifies and keeps the most relevant input features while removing irrelevant or redundant ones to improve model performance and reduce complexity.

Feature importance measures how much each input feature contributes to a model's predictions, helping understand which factors drive outcomes.

Normalization scales numerical features to a standard range, typically 0 to 1, ensuring no single feature dominates due to its scale.

Standardization transforms features to have zero mean and unit standard deviation, making them comparable regardless of original scale and distribution.

One-hot encoding converts categorical variables into binary vectors where each category becomes a separate binary feature with a value of 0 or 1.

SHAP values explain individual predictions by attributing the contribution of each feature based on Shapley values from cooperative game theory.

Adam is an adaptive learning rate optimizer that combines momentum and RMSprop to efficiently train deep learning models with per-parameter learning rates.

Cross-entropy loss measures the difference between predicted probability distributions and true labels, the standard loss function for classification tasks.

Mean squared error measures the average squared difference between predicted and actual values, the standard loss function for regression tasks.