|
SSJ V. 2.6.2. |
||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object umontreal.iro.lecuyer.gof.GofStat
public class GofStat
This class provides methods to compute several types of EDF goodness-of-fit test statistics and to apply certain transformations to a set of observations. This includes the probability integral transformation Ui = F(Xi), as well as the power ratio and iterated spacings transformations. Here, U(0),..., U(n-1) stand for n observations U0,..., Un-1 sorted by increasing order, where 0 <= Ui <= 1.
Note: This class uses the Colt library.
Nested Class Summary | |
---|---|
static class |
GofStat.OutcomeCategoriesChi2
This class helps managing the partitions of possible outcomes into categories for applying chi-square tests. |
Field Summary | |
---|---|
static double |
EPSILONAD
|
Method Summary | |
---|---|
static double |
andersonDarling(double[] sortedData)
Computes and returns the Anderson-Darling statistic An2. |
static double[] |
andersonDarling(double[] data,
ContinuousDistribution dist)
Computes the Anderson-Darling statistic An2 and the corresponding p-value p. |
static double |
andersonDarling(DoubleArrayList sortedData)
Computes and returns the Anderson-Darling statistic An2 (see method andersonDarling ). |
static double |
chi2(double[] nbExp,
int[] count,
int smin,
int smax)
Computes and returns the chi-square statistic for the observations oi in count[smin...smax], for which the corresponding expected values ei are in nbExp[smin...smax]. |
static double |
chi2(GofStat.OutcomeCategoriesChi2 cat,
int[] count)
Computes and returns the chi-square statistic for the observations oi in count, for which the corresponding expected values ei are in cat. |
static double |
chi2(IntArrayList data,
DiscreteDistributionInt dist,
int smin,
int smax,
double minExp,
int[] numCat)
Computes and returns the chi-square statistic for the observations stored in data, assuming that these observations follow the discrete distribution dist. |
static double |
chi2Equal(DoubleArrayList data)
Equivalent to chi2Equal (data, 10). |
static double |
chi2Equal(DoubleArrayList data,
double minExp)
Computes the chi-square statistic for a continuous distribution. |
static double |
chi2Equal(double nbExp,
int[] count,
int smin,
int smax)
Similar to chi2 ,
except that the expected
number of observations per category is assumed to be the same for
all categories, and equal to nbExp. |
static double |
cramerVonMises(DoubleArrayList sortedData)
Computes and returns the Cramér-von Mises statistic Wn2. |
static void |
diff(DoubleArrayList sortedData,
DoubleArrayList spacings,
int n1,
int n2,
double a,
double b)
Same as method diff (IntArrayList,IntArrayList,int,int,int,int),
but for the continuous case. |
static void |
diff(IntArrayList sortedData,
IntArrayList spacings,
int n1,
int n2,
int a,
int b)
Assumes that the real-valued observations U0,..., Un-1 contained in sortedData are already sorted in increasing order and computes the differences between the successive observations. |
static void |
iterateSpacings(DoubleArrayList data,
DoubleArrayList spacings)
Applies one iteration of the iterated spacings transformation. |
static double[] |
kolmogorovSmirnov(double[] sortedData)
Computes the Kolmogorov-Smirnov (KS) test statistics Dn+, Dn-, and Dn (see method kolmogorovSmirnov ). |
static void |
kolmogorovSmirnov(double[] data,
ContinuousDistribution dist,
double[] sval,
double[] pval)
Computes the KolmogorovSmirnov (KS) test statistics and their p-values. |
static double[] |
kolmogorovSmirnov(DoubleArrayList sortedData)
Computes the Kolmogorov-Smirnov (KS) test statistics Dn+, Dn-, and Dn. |
static double[] |
kolmogorovSmirnovJumpOne(DoubleArrayList sortedData,
double a)
Compute the KS statistics Dn+(a) and Dn-(a) defined in the description of the method FDist.kolmogorovSmirnovPlusJumpOne , assuming that F is the
uniform distribution over [0, 1] and that
U(1),..., U(n) are in sortedData. |
static double |
pDisc(double pL,
double pR)
Computes a variant of the p-value p whenever a test statistic has a discrete probability distribution. |
static void |
powerRatios(DoubleArrayList sortedData)
Applies the power ratios transformation W. |
static int |
scan(DoubleArrayList sortedData,
double d)
Computes and returns the scan statistic Sn(d ), defined in FBar.scan . |
static DoubleArrayList |
unifTransform(DoubleArrayList data,
ContinuousDistribution dist)
Applies the probability integral transformation Ui = F(Vi) for i = 0, 1,…, n - 1, where F is a continuous distribution function, and returns the result as an array of length n. |
static DoubleArrayList |
unifTransform(DoubleArrayList data,
DiscreteDistribution dist)
Applies the transformation Ui = F(Vi) for i = 0, 1,…, n - 1, where F is a discrete distribution function, and returns the result as an array of length n. |
static double |
watsonG(DoubleArrayList sortedData)
Computes and returns the Watson statistic Gn. |
static double |
watsonU(DoubleArrayList sortedData)
Computes and returns the Watson statistic Un2. |
Methods inherited from class java.lang.Object |
---|
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static double EPSILONAD
Method Detail |
---|
public static DoubleArrayList unifTransform(DoubleArrayList data, ContinuousDistribution dist)
data
- array of observations to be transformeddist
- assumed distribution of the observations
public static DoubleArrayList unifTransform(DoubleArrayList data, DiscreteDistribution dist)
Note: If V are the values of random variables with distribution function dist, then the result will contain the values of discrete random variables distributed over the set of values taken by dist, not uniform random variables over [0, 1].
data
- array of observations to be transformeddist
- assumed distribution of the observations
public static void diff(IntArrayList sortedData, IntArrayList spacings, int n1, int n2, int a, int b)
sortedData
- array of sorted observationsspacings
- pointer to an array object that will be filled with spacingsn1
- starting index, in sortedData, of the processed observationsn2
- ending index, in sortedData of the processed observationsa
- minimum value of the observationsb
- maximum value of the observationspublic static void diff(DoubleArrayList sortedData, DoubleArrayList spacings, int n1, int n2, double a, double b)
diff
(IntArrayList,IntArrayList,int,int,int,int),
but for the continuous case.
sortedData
- array of sorted observationsspacings
- pointer to an array object that will be filled with spacingsn1
- starting index of the processed observations in sortedDatan2
- ending index, in sortedData of the processed observationsa
- minimum value of the observationsb
- maximum value of the observationspublic static void iterateSpacings(DoubleArrayList data, DoubleArrayList spacings)
diff
.
This method transforms the spacings into new
spacings:
it sorts
S0,..., Sn to obtain
S(0) <= S(1) <= S(2) <= ... <= S(n),
computes the weighted differences
S0 | = | (n + 1)S(0), | |
S1 | = | n(S(1) - S(0)), | |
S2 | = | (n - 1)(S(2) - S(1)), | |
... | |||
Sn | = | S(n) - S(n-1), |
Under the assumption that the Ui are i.i.d. U(0, 1), the new Si can be considered as a new set of spacings having the same distribution as the original spacings, and the Vi are a new sample of i.i.d. U(0, 1) random variables, sorted by increasing order.
This transformation is useful to detect clustering in a data set: A pair of observations that are close to each other is transformed into an observation close to zero. A data set with unusually clustered observations is thus transformed to a data set with an accumulation of observations near zero, which is easily detected by the Anderson-Darling GOF test.
data
- array of observationsspacings
- spacings between the observations, will be filled with the new spacingspublic static void powerRatios(DoubleArrayList sortedData)
This transformation is useful to detect clustering, as explained in
iterateSpacings
,
except that here a pair of
observations close to each other is transformed
into an observation close to 1.
An accumulation of observations near 1 is also easily detected by
the Anderson-Darling GOF test.
sortedData
- sorted array of real-valued observations in the interval [0, 1]
that will be overwritten with the transformed observationspublic static double chi2(double[] nbExp, int[] count, int smin, int smax)
nbExp
- numbers expected in each categorycount
- numbers observed in each categorysmin
- index of the first valid data in count and nbExpsmax
- index of the last valid data in count and nbExp
public static double chi2(GofStat.OutcomeCategoriesChi2 cat, int[] count)
cat
- numbers expected in each categorycount
- numbers observed in each category
public static double chi2(IntArrayList data, DiscreteDistributionInt dist, int smin, int smax, double minExp, int[] numCat)
Generally, it is not possible to divide the integers in intervals satisfying
nP(a0 <= s < a1) = nP(a1 <= s < a2) = ... = nP(aj-1 <= s < aj)
for a discrete distribution, where n is the sample size, i.e.,
the number of
observations stored into data.
To perform a general chi-square test, the method starts
from smin and finds the first non-negligible
probability
p(s) >= ε, where
ε = DiscreteDistributionInt.EPSILON
.
It uses smax to allocate an array storing the
number of expected observations (np(s)) for each s >= smin.
Starting from s = smin, the np(s) terms are computed and
the allocated array grows if required until a negligible probability
term is found.
This gives the number of expected elements for
each category, where an outcome category corresponds here to
an interval in which sample observations could lie.
The categories are regrouped to have at least
minExp observations per category.
The method then counts the number of samples in each categories and calls
chi2
to get the chi-square test
statistic. If numCat is not
null, the number of categories after regrouping is returned
in numCat[0]. The number of degrees of freedom is equal to
numCat[0]-1. We usually choose minExp = 10.
data
- observations, not necessarily sorteddist
- assumed probability distributionsmin
- estimated minimum value of s for which p(s) > 0smax
- estimated maximum value of s for which p(s) > 0minExp
- minimum number of expected observations in each
intervalnumCat
- one-element array that will be filled with the number of
categories after regrouping
public static double chi2Equal(double nbExp, int[] count, int smin, int smax)
chi2
,
except that the expected
number of observations per category is assumed to be the same for
all categories, and equal to nbExp.
nbExp
- number of expected observations in each category (or interval)count
- number of counted observations in each categorysmin
- index of the first valid data in count and nbExpsmax
- index of the last valid data in count and nbExp
public static double chi2Equal(DoubleArrayList data, double minExp)
chi2Equal
.
We usually choose minExp = 10.
data
- array of observations in [0, 1)minExp
- minimum number of expected observations in each subintervals
public static double chi2Equal(DoubleArrayList data)
data
- array of observations in [0, 1)
public static int scan(DoubleArrayList sortedData, double d)
FBar.scan
.
Let U be the n observations contained into sortedData.
The n observations in U[0..n - 1] must be real numbers
in the interval [0, 1], sorted in increasing order.
(See FBar.scan
for the distribution function of Sn(d )).
sortedData
- sorted array of real-valued observations in the interval [0, 1]d
- length of the test interval (∈(0, 1))
public static double cramerVonMises(DoubleArrayList sortedData)
sortedData
- array of sorted real-valued observations in the interval [0, 1]
public static double watsonG(DoubleArrayList sortedData)
Gn | = | (n)1/2max0 <= j <= n-1{(j + 1)/n - U(j) + bar(U)n -1/2} | |
= | (n)1/2(Dn+ + bar(U)n - 1/2), |
sortedData
- array of sorted real-valued observations in the interval [0, 1]
public static double watsonU(DoubleArrayList sortedData)
Wn2 | = | 1/(12n) + ∑j=0n-1{U(j) - (j + 0.5)/n}2, | |
Un2 | = | Wn2 - n(bar(U)n -1/2)2. |
sortedData
- array of sorted real-valued observations in the interval [0, 1]
public static double andersonDarling(DoubleArrayList sortedData)
andersonDarling
).
sortedData
- array of sorted real-valued observations in the interval [0, 1]
public static double andersonDarling(double[] sortedData)
An2 | = | - n - 1/n ∑j=0n-1{(2j + 1)ln(U(j)) + (2n - 1 - 2j)ln(1 - U(j))}, |
sortedData
- array of sorted real-valued observations in the interval [0, 1]
public static double[] andersonDarling(double[] data, ContinuousDistribution dist)
data
- array of observationsdist
- assumed distribution of the observations
public static double[] kolmogorovSmirnov(double[] sortedData)
kolmogorovSmirnov
). Returns the array [Dn+, Dn-, Dn].
sortedData
- array of sorted real-valued observations in the interval [0, 1]
public static double[] kolmogorovSmirnov(DoubleArrayList sortedData)
Dn+ | = | max0 <= j <= n-1((j + 1)/n - U(j)), | |
Dn- | = | max0 <= j <= n-1(U(j) - j/n), | |
Dn | = | max (Dn+, Dn-). |
sortedData
- array of sorted real-valued observations in the interval [0, 1]
public static void kolmogorovSmirnov(double[] data, ContinuousDistribution dist, double[] sval, double[] pval)
data
- array of observations to be testeddist
- assumed distribution of the observationssval
- values of the 3 KS statisticspval
- p-values for the 3 KS statisticspublic static double[] kolmogorovSmirnovJumpOne(DoubleArrayList sortedData, double a)
FDist.kolmogorovSmirnovPlusJumpOne
, assuming that F is the
uniform distribution over [0, 1] and that
U(1),..., U(n) are in sortedData.
Returns the array [Dn+, Dn-].
sortedData
- array of sorted real-valued observations in the interval [0, 1]a
- size of the jump
public static double pDisc(double pL, double pR)
pL | = | P[Y <= y] | |
pR | = | P[Y >= y] |
p = | pR, | if pR < pL, |
p = | 1 - pL, | if pR >= pL and pL < 0.5, |
p = | 0.5 | otherwise. |
pL
- left p-valuepR
- right p-value
|
SSJ V. 2.6.2. |
||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |