Feel the vibe, feel the terror, feel the painMad about you, orchestrally.more quotes

# statistics: fun

DNA on 10th — street art, wayfinding and font

# Nature Methods: Points of Significance

Points of Significance column in Nature Methods. (Launch of Points of Significance)
46 | Grewal, J., Krzywinski, M. & Altman, N. (2019) Points of significance: Hidden Markov models. Nature Methods 16:795–796.
45 | Grewal, J., Krzywinski, M. & Altman, N. (2019) Points of significance: Markov chains. Nature Methods 16:663–664.
44 | Das, K., Krzywinski, M. & Altman, N. (2019) Points of significance: Quantile regression. Nature Methods 16:451–452.
43 | Greco, L., Luta, G., Krzywinski, M. & Altman, N. (2019) Points of significance: Analyzing outliers: Robust methods to the rescue. Nature Methods 16:275–276.
42 | Smucker, B., Krzywinski, M. & Altman, N. (2019) Points of significance: Two-level factorial experiments Nature Methods 16:211–212.
41 | Altman, N. & Krzywinski, M. (2018) Points of significance: Predicting with confidence and tolerance Nature Methods 15:843–844.
40 | Smucker, B., Krzywinski, M. & Altman, N. (2018) Points of significance: Optimal experimental design Nature Methods 15:559–560.
39 | Altman, N. & Krzywinski, M. (2018) Points of significance: Curse(s) of dimensionality Nature Methods 15:299–400.
38 | Bzdok, D., Krzywinski, M. & Altman, N. (2018) Points of significance: Statistics vs machine learning. Nature Methods 15:233–234.
37 | Bzdok, D., Krzywinski, M. & Altman, N. (2018) Points of significance: Machine learning: supervised methods. Nature Methods 15:5–6.
36 | Bzdok, D., Krzywinski, M. & Altman, N. (2017) Points of significance: Machine learning: a primer. Nature Methods 14:1119–1120.
35 | Altman, N. & Krzywinski, M. (2017) Points of significance: Ensemble methods: Bagging and random forests. Nature Methods 14:933–934.
34 | Krzywinski, M. & Altman, N. (2017) Points of significance: Classification and regression trees. Nature Methods 14:757–758.
33 | Lever, J., Krzywinski, M. & Altman, N. (2017) Points of significance: Principal component analysis. Nature Methods 14:641–642.
32 | Altman, N. & Krzywinski, M. (2017) Points of significance: Clustering. Nature Methods 14:545–546.
31 | Altman, N. & Krzywinski, M. (2017) Points of significance: Tabular data. Nature Methods 14:329–330.
30 | Altman, N. & Krzywinski, M. (2017) Points of significance: Interpreting P values. Nature Methods 14:213–214.
29 | Altman, N. & Krzywinski, M. (2017) Points of significance: P values and the search for significance. Nature Methods 14:3–4.
28 | Lever, J., Krzywinski, M. & Altman, N. (2016) Points of significance: Regularization. Nature Methods 13:803–804.
27 | Lever, J., Krzywinski, M. & Altman, N. (2016) Points of significance: Model selection and overfitting. Nature Methods 13:703–704.
26 | Lever, J., Krzywinski, M. & Altman, N. (2016) Points of significance: Classifier evaluation. Nature Methods 13:603–604.
25 | Lever, J., Krzywinski, M. & Altman, N. (2016) Points of significance: Logistic regression. Nature Methods 13:541–542.
24 | Altman, N. & Krzywinski, M. (2016) Points of significance: Regression diagnostics. Nature Methods 13:385–386.
23 | Altman, N. & Krzywinski, M. (2016) Points of significance: Analyzing outliers: Influential or nuisance. Nature Methods 13:281–282.
22 | Krzywinski, M. & Altman, N. (2015) Points of significance: Multiple linear regression. Nature Methods 12:1103–1104.
21 | Altman, N. & Krzywinski, M. (2015) Points of significance: Simple linear regression. Nature Methods 12:999–1000.
20 | Altman, N. & Krzywinski, M. (2015) Points of significance: Association, correlation and causation. Nature Methods 12:899–900.
19 | Puga, J.L, Krzywinski, M. & Altman, N. (2015) Points of significance: Bayesian networks. Nature Methods 12:799–800.
18 | Kulesa, A., Krzywinski, M., Blainey, P. & Altman, N. (2015) Points of significance: Sampling distributions and the bootstrap. Nature Methods 12:477–478.
17 | Puga, J.L, Krzywinski, M. & Altman, N. (2015) Points of significance: Bayesian statistics. Nature Methods 12:277–278.
16 | Puga, J.L, Krzywinski, M. & Altman, N. (2015) Points of significance: Bayes' theorem. Nature Methods 12:277–278.
15 | Altman, N. & Krzywinski, M. (2015) Points of significance: Split plot design. Nature Methods 12:165–166.
14 | Altman, N. & Krzywinski, M. (2015) Points of significance: Sources of variation. Nature Methods 12:5–6.
13 | Krzywinski, M., Altman, N. (2014) Points of significance: Two factor designs. Nature Methods 11:1187–1188.
12 | Krzywinski, M., Altman, N. & Blainey, P. (2014) Points of significance: Nested designs. Nature Methods 11:977–978.
11 | Blainey, P., Krzywinski, M. & Altman, N. (2014) Points of significance: Replication. Nature Methods 11:879–880.
10 | Krzywinski, M. & Altman, N. (2014) Points of significance: Analysis of variance (ANOVA) and blocking. Nature Methods 11:699–700.
9 | Krzywinski, M. & Altman, N. (2014) Points of significance: Designing comparative experiments. Nature Methods 11:597–598.
8 | Krzywinski, M. & Altman, N. (2014) Points of significance: Non-parametric tests. Nature Methods 11:467–468.
7 | Krzywinski, M. & Altman, N. (2014) Points of significance: Comparing samples — Part II — Multiple testing. Nature Methods 11:355–356.
6 | Krzywinski, M. & Altman, N. (2014) Points of significance: Comparing samples — Part I — t–tests. Nature Methods 11:215–216.
5 | Krzywinski, M. & Altman, N. (2014) Points of significance: Visualizing samples with box plots. Nature Methods 11:119–120.
4 | Krzywinski, M. & Altman, N. (2013) Points of significance: Power and sample size. Nature Methods 10:1139–1140.
3 | Krzywinski, M. & Altman, N. (2013) Points of significance: Significance, P values and t–tests. Nature Methods 10:1041–1042.
2 | Krzywinski, M. & Altman, N. (2013) Points of significance: Error bars. Nature Methods 10:921–922.
1 | Krzywinski, M. & Altman, N. (2013) Points of significance: Importance of being uncertain. Nature Methods 10:809–810.
VIEW ALL

# Scientific data visualization: Aesthetic for diagrammatic clarity

Mon 13-01-2020

The scientific process works because all its output is empirically constrained.

My chapter from The Aesthetics of Scientific Data Representation, More than Pretty Pictures, in which I discuss the principles of data visualization and connect them to the concept of "quality" introduced by Robert Pirsig in Zen and the Art of Motorcycle Maintenance.

# Yearning for the Infinite — Aleph 2

Mon 18-11-2019

Discover Cantor's transfinite numbers through my music video for the Aleph 2 track of Max Cooper's Yearning for the Infinite (album page, event page).

Yearning for the Infinite, Max Cooper at the Barbican Hall, London. Track Aleph 2. Video by Martin Krzywinski. Photo by Michal Augustini. (more)

I discuss the math behind the video and the system I built to create the video.

# Hidden Markov Models

Mon 18-11-2019

Everything we see hides another thing, we always want to see what is hidden by what we see.
—Rene Magritte

A Hidden Markov Model extends a Markov chain to have hidden states. Hidden states are used to model aspects of the system that cannot be directly observed and themselves form a Markov chain and each state may emit one or more observed values.

Hidden states in HMMs do not have to have meaning—they can be used to account for measurement errors, compress multi-modal observational data, or to detect unobservable events.

Nature Methods Points of Significance column: Hidden Markov Models. (read)

In this column, we extend the cell growth model from our Markov Chain column to include two hidden states: normal and sedentary.

We show how to calculate forward probabilities that can predict the most likely path through the HMM given an observed sequence.

Grewal, J., Krzywinski, M. & Altman, N. (2019) Points of significance: Hidden Markov Models. Nature Methods 16:795–796.

Altman, N. & Krzywinski, M. (2019) Points of significance: Markov Chains. Nature Methods 16:663–664.

# Hola Mundo Cover

Sat 21-09-2019

My cover design for Hola Mundo by Hannah Fry. Published by Blackie Books.

Hola Mundo by Hannah Fry. Cover design is based on my 2013 $\pi$ day art. (read)

Curious how the design was created? Read the full details.

# Markov Chains

Tue 30-07-2019

You can look back there to explain things,
but the explanation disappears.
You'll never find it there.
Things are not explained by the past.
They're explained by what happens now.
—Alan Watts

A Markov chain is a probabilistic model that is used to model how a system changes over time as a series of transitions between states. Each transition is assigned a probability that defines the chance of the system changing from one state to another.

Nature Methods Points of Significance column: Markov Chains. (read)

Together with the states, these transitions probabilities define a stochastic model with the Markov property: transition probabilities only depend on the current state—the future is independent of the past if the present is known.

Once the transition probabilities are defined in matrix form, it is easy to predict the distribution of future states of the system. We cover concepts of aperiodicity, irreducibility, limiting and stationary distributions and absorption.

This column is the first part of a series and pairs particularly well with Alan Watts and Blond:ish.

Grewal, J., Krzywinski, M. & Altman, N. (2019) Points of significance: Markov Chains. Nature Methods 16:663–664.

# 1-bit zoomable gigapixel maps of Moon, Solar System and Sky

Mon 22-07-2019

Places to go and nobody to see.

Exquisitely detailed maps of places on the Moon, comets and asteroids in the Solar System and stars, deep-sky objects and exoplanets in the northern and southern sky. All maps are zoomable.

3.6 gigapixel map of the near side of the Moon, annotated with 6,733. (details)
100 megapixel and 10 gigapixel map of the Solar System on 20 July 2019, annotated with 758k asteroids, 1.3k comets and all planets and satellites. (details)
100 megapixle and 10 gigapixel map of the Northern Celestial Hemisphere, annotated with 44 million stars, 74,000 deep-sky objects and 3,000 exoplanets. (details)
100 megapixle and 10 gigapixel map of the Southern Celestial Hemisphere, annotated with 69 million stars, 88,000 deep-sky objects and 1000 exoplanets. (details)

# Quantile regression

Sat 01-06-2019
Quantile regression robustly estimates the typical and extreme values of a response.

Quantile regression explores the effect of one or more predictors on quantiles of the response. It can answer questions such as "What is the weight of 90% of individuals of a given height?"

Nature Methods Points of Significance column: Quantile regression. (read)

Unlike in traditional mean regression methods, no assumptions about the distribution of the response are required, which makes it practical, robust and amenable to skewed distributions.

Quantile regression is also very useful when extremes are interesting or when the response variance varies with the predictors.

Das, K., Krzywinski, M. & Altman, N. (2019) Points of significance: Quantile regression. Nature Methods 16:451–452.

Altman, N. & Krzywinski, M. (2015) Points of significance: Simple linear regression. Nature Methods 12:999–1000.

# Analyzing outliers: Robust methods to the rescue

Sat 30-03-2019
Robust regression generates more reliable estimates by detecting and downweighting outliers.

Outliers can degrade the fit of linear regression models when the estimation is performed using the ordinary least squares. The impact of outliers can be mitigated with methods that provide robust inference and greater reliability in the presence of anomalous values.

Nature Methods Points of Significance column: Analyzing outliers: Robust methods to the rescue. (read)

We discuss MM-estimation and show how it can be used to keep your fitting sane and reliable.

Greco, L., Luta, G., Krzywinski, M. & Altman, N. (2019) Points of significance: Analyzing outliers: Robust methods to the rescue. Nature Methods 16:275–276.