A practical and conceptual study of Bayesian filtering and smoothing for systems whose internal states are not directly observable and evolve over time under uncertainty.
The emphasis is on inference, not simulation: how noisy measurements and imperfect models are combined to reason about hidden states, and how uncertainty propagates through time.
In robotics and dynamical systems, the true state of a system is rarely known exactly. Instead, we observe noisy sensor data and rely on approximate models of system dynamics. Bayesian filtering and smoothing provide a principled framework for state estimation under uncertainty, where the system state is treated as a random variable rather than a deterministic quantity.
This repository builds these ideas step by step, starting from linear Gaussian models and Bayesian regression, and progressing toward recursive Bayesian filters and particle-based methods. Emphasis is placed on understanding:
- what is uncertain,
- where that uncertainty comes from,
- how it propagates through time,
- and why filtering methods succeed or fail in practice.
The material is developed through formal derivations (PDFs), executable notebooks, and diagnostic visualizations, closely tied to probabilistic modeling foundations.
- Linear Gaussian estimation and covariance inference
- Batch vs recursive Bayesian regression
- Kalman Filter (KF)
- Extended Kalman Filter (EKF)
- Unscented Kalman Filter (UKF)
- Gauss–Hermite Kalman Filter (GHKF)
- Particle Filtering (bootstrap and EKF/UKF-based proposals)
- Importance sampling, weight degeneracy, ESS, and resampling
- Practical diagnostics and failure modes
The following animations show Bayesian state estimation on a nonlinear pendulum, comparing different Gaussian filtering methods.
Each animation overlays the true system trajectory, noisy measurements, and the filter’s estimated state evolution.
State estimation for a nonlinear pendulum using EKF. Linearization error and approximation effects are visible.
GHKF with order 3 (left) and order 5 (right), showing improved nonlinear moment approximation with higher quadrature order.
UKF state estimation using sigma-point propagation for nonlinear dynamics.
These visualizations are generated directly from the filtering implementations in src/filters and are used to study approximation quality, consistency, and failure modes across methods.
- Kalman Filter
- Extended Kalman Filter
- Gauss–Hermite Kalman Filter
- Unscented Kalman Filter
- Particle Filter
Formal derivations and conceptual write-ups supporting the implementations:
01_linear_gaussian_estimation_covariance_inference.pdf02_Linear_Gaussian_Regression_Batch_Recursive.pdf03_Bayesian_modeling_for_Dynamical_Systems.pdf04_Bayesian_Filtering_Equations.pdf05_Gaussian_Filtering_Equations.pdf06_Kalman_Filtering_Equations.pdf07_Extended_Kalman_Filtering_Equations.pdf10_Gaussian_Filtering_Summary_and_motivation_for_ParticleFiltering.pdf11_PF_Empirical_Distributions_and_Importance_Sampling_Foundations.pdf12_Sequential_Self-Normalized_Importance_Sampling.pdf13_WeightDegeneracy_ESS_Resampling.pdf15_Optimal_Importance_Proposal_for_ParticleFiltering.pdf16_GaussianApproximations_of_the_OptimalProposal_in_ParticleFiltering.pdf17_OptimalProposal_for_LinearMeasurementModel_in_ParticleFiltering.pdf
Bayesian-Filtering-and-Smoothing/
│
├── images/
│ ├── bayesian_filtering/
│ └── regression/
│
├── notebooks/
│ ├── ch01_regression/
│ └── ch02_filtering/
│
├── reports/
│
├── src/
│ ├── regression/
│ │ └── linear_regression.py
│ │
│ ├── filters/
│ │ ├── kalman_filter.py
│ │ ├── extended_kalman_filter.py
│ │ ├── unscented_kalman_filter.py
│ │ ├── gauss_hermite_kalman_filter.py
│ │ ├── particle_filter.py
│ │ ├── bootstrap_particle_filter.py
│ │ ├── ekf_particle_filter.py
│ │ └── ukf_particle_filter.py
│ │
│ └── models/
│ └── pendulum.py
│
├── README.md
└── pyproject.toml
🚧 Under active development
Further extensions include smoother implementations, deeper diagnostic studies, and tighter integration between Gaussian and particle-based methods.
I study and implement methods in optimization, control, robotics, Bayesian inference, and probabilistic reasoning.