Tutorial for time series ordinal pattern tests

Libraries

using StatsOrdinalPatterns, Random, Statistics, DataFrames, CairoMakie

Hypothesis Testing with Ordinal Patterns

An ordinal pattern test asks whether the frequencies of rank permutations in a time series are consistent with an i.i.d. (white noise) process. For a series \(x_1, \ldots, x_n\), overlapping windows of length \(m\) are each mapped to the permutation that describes the relative order of the values. The resulting pattern frequency distribution is summarised by a scalar test statistic. Under \(H_0\) (i.i.d.) that distribution is uniform across all \(m!\) patterns, so the statistic has a known or bootstrap-approximated null distribution.

Key functions

The key functions in StatsOrdinalPatterns.jl for testing time series regarding dependence using ordinal patterns are:

Computing the statistic
- stat_op(data; chart_choice, m=3, d=1) — computes the chosen test statistic from the observed pattern frequencies.
  - Returns [test_statistic, pattern_frequencies].
  - chart_choice: one of
  - m: pattern length (window size)
  - d: delay between observations within a window.
Asymptotic inference
- test_op(data; chart_choice, m=3, d=1, alpha=0.05)
  - Returns an OPTestResult with the test statistic, critical value, p-value, and reject decision.
  - Critical value, p-value, and rejection direction are chosen automatically from chart_choice.
  - alpha: significance level.
  - Relies on asymptotic limiting distributions; currently supports m = 2 and m = 3 only.
Bootstrap inference
- bootstrap_op(data, n_boot; chart_choice, m=3, d=1, block_size=1)
  - Returns a length-n_boot vector of resampled statistics under \(H_0\).
  - n_boot: number of bootstrap replications.
  - block_size = 1: standard i.i.d. bootstrap. block_size > 1: block bootstrap that preserves local serial structure.
  - Works for any m.
- test_op_bootstrap(data, n_boot; chart_choice, m=3, d=1, alpha=0.05, block_size=1)
  - Returns an OPTestResultBoot struct with the observed statistic, bootstrap critical value, bootstrap p-value, and reject decision.
  - Handles the correct bootstrap direction (two-sided, upper-tail, lower-tail) automatically.
  - alpha: significance level. block_size: as in bootstrap_op().
  - Works for any m — the only inference option when m > 3.

Available chart statistics

Seven chart statistics are available. They differ in what they measure, the direction of the rejection region, and the limiting distribution under \(H_0\).

Statistic	Constructor	Under \(H_0\)	Reject \(H_0\) when	Distribution
Persistence (\(\hat{\tau}\))	`Persistence()`	\(\hat{\tau} = 0\)	\(\lvert\hat{\tau}\rvert > c\)	Normal
UpDownBalance (\(\hat{\beta}\))	`UpDownBalance()`	\(\hat{\beta} = 0\)	\(\lvert\hat{\beta}\rvert > c\)	Normal
RotationalAsymmetry (\(\hat{\gamma}\))	`RotationalAsymmetry()`	\(\hat{\gamma} = 0\)	\(\lvert\hat{\gamma}\rvert > c\)	Normal
UpDownScaling (\(\hat{\delta}\))	`UpDownScaling()`	\(\hat{\delta} = 0\)	\(\lvert\hat{\delta}\rvert > c\)	Normal
DistanceToWhiteNoise (\(\hat{\Delta}\))	`DistanceToWhiteNoise()`	\(\hat{\Delta} = 0\)	\(\hat{\Delta} > c\)	Generalized \(\chi^2\)
Shannon entropy (\(\hat{H}\))	`Shannon(base=exp(1))`	\(\hat{H} = \log(m!)\)	\(\hat{H} < c\)	Generalized \(\chi^2\)
Shannon extropy (\(\hat{H}_\text{ex}\))	`ShannonExtropy(base=exp(1))`	\(\hat{H}_\text{ex} = 5\log(6/5)\)	\(\hat{H}_\text{ex} < c\)	Generalized \(\chi^2\)

The first four statistics use a two-sided test. DistanceToWhiteNoise is always non-negative — upper-tail.
The entropy statistics decrease under non-uniformity — lower-tail.
The test direction determines how bootstrap p-values are computed (see below).
The asymptotic critical values and their limiting distributions are derived in Weiß (2022).

Logarithm base

Shannon and ShannonExtropy come from ComplexityMeasures.jl, where both use base 2 by default. The papers behind this package use the natural logarithm, and all asymptotic results are derived for it. Write Shannon(base=exp(1)) to obtain the values reported in the papers. The Unicode form Shannon(base=ℯ), typed as \euler followed by Tab, is equivalent.

Any base larger than 1 is allowed. An entropy in base \(b\) equals the entropy under the natural logarithm divided by \(\ln b\). The package applies this conversion as follows.

Quantity	With base \(b\)
Statistic of `stat_op`, `stat_sop` and `test_op`	divided by \(\ln b\)
Critical value of `test_op` and `crit_val_op`	divided by \(\ln b\)
Control limit `cl` of the EWMA charts	must be given in base \(b\)
Box-Pierce statistic and rescaled SOP statistic	unchanged
p-value and test decision	unchanged

For example, the pattern frequencies \(\hat{p} = (0.30, 0.20, 0.15, 0.15, 0.10, 0.10)\) give a Shannon entropy of 1.7127 under the natural logarithm, \(1.7127 / \ln 2 = 2.4710\) in base 2, and \(1.7127 / \ln 10 = 0.7438\) in base 10.

p = [0.30, 0.20, 0.15, 0.15, 0.10, 0.10]
[chart_stat_op(p, Shannon(base=b)) for b in (exp(1), 2, 10)]

3-element Vector{Float64}:
 1.7127324378491742
 2.4709505944546684
 0.7438302467346005

A base of 1 or smaller raises an error. Base 1 would divide by zero, and a base between 0 and 1 would make the entropy negative and reverse the direction of the tests.

Setup

Random.seed!(42)
n, m, d = 300, 3, 1
n_pat = n - (m - 1) * d

Example data

data_iid = randn(n)   # i.i.d. — H₀ is true

φ = 0.5
data_ar1 = zeros(n);
data_ar1[1] = randn();
for t in 2:n
    data_ar1[t] = φ * data_ar1[t-1] + randn()
end   # AR(1) — H₀ is false

Asymptotic testing: `test_op()`

Two-sided — Persistence

test_op(data_iid; chart_choice=Persistence(), m=m, d=d)

OPTestResult
  Chart:            Persistence()
  Statistic:        0.0157
  ─────────────────────────────
  Asymptotic test
    Critical value: 0.0479
    p-value:        0.5214
    Reject H₀:      false

test_op(data_ar1; chart_choice=Persistence(), m=m, d=d)

OPTestResult
  Chart:            Persistence()
  Statistic:        0.123
  ─────────────────────────────
  Asymptotic test
    Critical value: 0.0479
    p-value:        0.0
    Reject H₀:      true

Upper-tail — DistanceToWhiteNoise

test_op(data_iid; chart_choice=DistanceToWhiteNoise(), m=m, d=d)

OPTestResult
  Chart:            DistanceToWhiteNoise()
  Statistic:        0.002
  ─────────────────────────────
  Asymptotic test
    Critical value: 0.005
    p-value:        0.3652
    Reject H₀:      false

test_op(data_ar1; chart_choice=DistanceToWhiteNoise(), m=m, d=d)

OPTestResult
  Chart:            DistanceToWhiteNoise()
  Statistic:        0.0114
  ─────────────────────────────
  Asymptotic test
    Critical value: 0.005
    p-value:        0.0011
    Reject H₀:      true

Lower-tail — Shannon entropy

test_op(data_iid; chart_choice=Shannon(base=exp(1)), m=m, d=d)

OPTestResult
  Chart:            Shannon(base = 2.718281828459045)
  Statistic:        1.7859
  ─────────────────────────────
  Asymptotic test
    Critical value: 1.7768
    p-value:        0.3679
    Reject H₀:      false

test_op(data_ar1; chart_choice=Shannon(base=exp(1)), m=m, d=d)

OPTestResult
  Chart:            Shannon(base = 2.718281828459045)
  Statistic:        1.759
  ─────────────────────────────
  Asymptotic test
    Critical value: 1.7768
    p-value:        0.0015
    Reject H₀:      true

The i.i.d. series is not rejected; the AR(1) series is. The critical value is the same for both series — it depends only on n_pat, m, and alpha, not on the data.

Bootstrap testing: `test_op_bootstrap()`

test_op_bootstrap() generates the empirical null distribution via bootstrap_op(), then computes the bootstrap critical value and p-value in the correct direction for the chosen statistic. It works for any m — including m > 3 where test_op() has no asymptotic theory.

Random.seed!(42)
res_boot = test_op_bootstrap(data_ar1, 2_000; chart_choice=Persistence(), m=m, d=d)

OPTestResultBoot
  Chart:            Persistence()
  Statistic:        0.123
  ─────────────────────────────
  Bootstrap  (n_boot = 2000)
    Critical value: 0.0481
    p-value:        0.0
    Reject H₀:      true

The bootstrap critical value agrees closely with the asymptotic one shown above. The table below confirms this across all seven statistics.

The result keeps the resampled statistics in res_boot.boot_dist, and with CairoMakie loaded plot draws them as the bootstrap null distribution, with the rejection region shaded, the critical values dashed and the observed statistic as a solid line. See the plotting tutorial for the options.

plot(res_boot)

Full comparison: asymptotic vs. bootstrap

Random.seed!(42)
chart_list = [
    ("Persistence (τ)", Persistence(), "two-sided"),
    ("UpDownBalance (β)", UpDownBalance(), "two-sided"),
    ("RotationalAsymmetry (γ)", RotationalAsymmetry(), "two-sided"),
    ("UpDownScaling (δ)", UpDownScaling(), "two-sided"),
    ("DistanceToWhiteNoise (Δ)", DistanceToWhiteNoise(), "upper-tail"),
    ("Shannon entropy (H)", Shannon(base=exp(1)), "lower-tail"),
    ("Shannon extropy (Hex)", ShannonExtropy(base=exp(1)), "lower-tail"),
]
asymp_res = [test_op(data_ar1; chart_choice=c, m=m, d=d) for (_, c, _) in chart_list]
boot_res = [test_op_bootstrap(data_ar1, 2_000; chart_choice=c, m=m, d=d) for (_, c, _) in chart_list]

DataFrame(
    Statistic=[t[1] for t in chart_list],
    Direction=[t[3] for t in chart_list],
    stat=round.([r.stat for r in asymp_res], digits=3),
    asymp_crit=round.([r.asymp_crit for r in asymp_res], digits=3),
    asymp_pval=round.([r.asymp_pval for r in asymp_res], digits=3),
    boot_crit=round.([r.boot_crit for r in boot_res], digits=3),
    boot_pval=round.([r.boot_pval for r in boot_res], digits=3),
    reject=[r.asymp_reject for r in asymp_res],
)

7×8 DataFrame

Row	Statistic	Direction	stat	asymp_crit	asymp_pval	boot_crit	boot_pval	reject
	String	String	Float64	Float64	Float64	Float64	Float64	Bool
1	Persistence (τ)	two-sided	0.123	0.048	0.0	0.048	0.0	true
2	UpDownBalance (β)	two-sided	-0.007	0.066	0.841	0.064	0.882	false
3	RotationalAsymmetry (γ)	two-sided	0.007	0.072	0.855	0.074	0.871	false
4	UpDownScaling (δ)	two-sided	-0.013	0.093	0.777	0.094	0.796	false
5	DistanceToWhiteNoise (Δ)	upper-tail	0.011	0.005	0.001	0.005	0.001	true
6	Shannon entropy (H)	lower-tail	1.759	1.777	0.001	1.777	0.002	true
7	Shannon extropy (Hex)	lower-tail	0.905	0.909	0.001	0.909	0.001	true

Across all seven statistics the analytical and bootstrap critical values are close, and the bootstrap p-values are consistent with the reject column — confirming that bootstrap_op() is a reliable alternative whenever the limiting distribution is complex or the sample size is small.

References

Weiß, Christian H. 2022. “Non-Parametric Tests for Serial Dependence in Time Series Based on Asymptotic Implementations of Ordinal-Pattern Statistics.” Chaos: An Interdisciplinary Journal of Nonlinear Science 32 (9). https://pubs.aip.org/aip/cha/article-abstract/32/9/093107/2835852/Non-parametric-tests-for-serial-dependence-in-time?redirectedFrom=fulltext.