Plackett–Burman designs are experimental designs presented in 1946 by Robin L. Plackett and J. P. Burman while working in the British Ministry of Supply.[1] Their goal was to find experimental designs for investigating the dependence of some measured quantity on a number of independent variables (factors), each taking L levels, in such a way as to minimize the variance of the estimates of these dependencies using a limited number of experiments. Interactions between the factors were considered negligible. The solution to this problem is to find an experimental design where each combination of levels for any pair of factors appears the same number of times, throughout all the experimental runs (refer to table). A complete factorial design would satisfy this criterion, but the idea was to find smaller designs.
| Run | X1 | X2 | X3 | X4 | X5 | X6 | X7 | X8 | X9 | X10 | X11 | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 | ||||||||||||
| 2 | - | - | - | - | - | - | ||||||
| 3 | - | - | - | - | - | - | ||||||
| 4 | - | - | - | - | - | - | ||||||
| 5 | - | - | - | - | - | - | ||||||
| 6 | - | - | - | - | - | - | ||||||
| 7 | - | - | - | - | - | - | ||||||
| 8 | - | - | - | - | - | - | ||||||
| 9 | - | - | - | - | - | - | ||||||
| 10 | - | - | - | - | - | - | ||||||
| 11 | - | - | - | - | - | - | ||||||
| 12 | - | - | - | - | - | - |
For the case of more than two levels, Plackett and Burman rediscovered designs that had previously been given by Raj Chandra Bose and K. Kishen at the Indian Statistical Institute.[4] Plackett and Burman give specifics for designs having a number of experiments equal to the number of levels L to some integer power, for L = 3, 4, 5, or 7.
When interactions between factors are not negligible, they are often confounded in Plackett–Burman designs with the main effects, meaning that the designs do not permit one to distinguish between certain main effects and certain interactions. This is called aliasing or confounding.
In 1993, Dennis Lin described a construction method via half-fractions of Plackett–Burman designs, using one column to take half of the rest of the columns.[5] The resulting matrix, minus that column, is a "supersaturated design"[6] for finding significant first order effects, under the assumption that few exist.
Box–Behnken designs can be made smaller, or very large ones constructed, by replacing the fractional factorials and incomplete blocks traditionally used for plan and seed matrices, respectively, with Plackett–Burmans. For example, a quadratic design for 30 variables requires a 30 column PB plan matrix of zeroes and ones, replacing the ones in each line using PB seed matrices of −1s and +1s (for 15 or 16 variables) wherever a one appears in the plan matrix, creating a 557 runs design with values, −1, 0, +1, to estimate the 496 parameters of a full quadratic model. Adding axial points allows estimating univariate cubic and quartic effects.
By equivocating certain columns with parameters to be estimated, Plackett–Burmans can also be used to construct mixed categorical and numerical designs, with interactions or high order effects, requiring no more than 4 runs more than the number of model parameters to be estimated. Sort by a-1 columns assigned to categorical variable A and following columns, where A = 1 + int(a·i /(max(i) + 0.00001)), i = row number and a = A's number of values. Next sort on columns assigned to any other categorical variables and following columns, repeating as needed. Such designs, if large, may otherwise be incomputable by standard search techniques like D-optimality. For example, 13 variables averaging 3 values each could have well over a million combinations to search. To estimate the 105 parameters in a quadratic model of 13 variables, one must formally exclude from consideration or compute |X'X| for well over 106C102, i.e. 313C105, or roughly 10484 matrices.
P.B.4 + + + + – – – + – – – + P.B.8 + + + + + + + + + – – – – + + – + + – – – + – – – + + – – + + – + – – – + – + – + – – – + – – + + – – – + + – + P.B.12 + + + + + + + + + + + + + + + – – – + – – – + + – – – + – – + – + + – + – + + + – – – – + – – + – – + – + + – + – – – + – – + – + + – + + – – – + – – + + – + – + + + – – – + – – + – – + – + + + – – – – + + + – – – + – + – – + – – + – + + + – – – – + – + + + – – + P.B.16 + + + + + + + + + + + + + + + + + + – – – – – – – – + + + + + + – + – – – – + + + – – – + + + – – + + + + – – – – – – + + – + + + – – + + – – + – – – + – + – – + + – – + + + – – – + – – + – + + – + – – – + + – + – – – + – – + – + + – + + – – + + + – + – + – + – – + – – – + + – + – + – + – + – + – – – + – + + – + – – + – + – + – – + – – – + – + + – + + – + – – – + + – – + + – – + – – + + – – + – + + – – + + – – – + + – – – + + + – – – – + + + – + – – – – – – + + + + – + + – + P.B.20 + + + + + + + + + + + + + + + + + + + + + + – + – + – – – – + + – – + – – + + + – + – + – – – – + + – – + – – + + + + – + – – – – + + – – + – – + + + – + + – – – – + + – – + – – + + + + – – + – + + + + – + – + – – – – + + – – – + – + – + – – – – + + – – + – – + + + + – + – – + + + + – + – + – – – – + – + – – + – – + + + + – + – + – – – – + + – – – + + – – + – – + + + + – + – – – + + + + – + – + – – – – + + – – + – – + + + – + – + – – – – + + – – + – + – + + – – + – – + + + + – + – + – – – – + – – + + + + – + – + – – – – + + – – + – – + – – + + + + – + – + – – – + – – + + – – + – – + + + + – + – + – – – – + – – – – + + – – + – – + + + + + – – – + + + + – + – + – – – – + + – + – – – + + – – + – – + + + + – + – + – – – – – – + + – – + – – + + + + – + + P.B.24 + + + + + + + + + + + + + + + + + + + + + + + + + + – + – + + – – + + – – + – + – – – – – + + + + – – + + – – + – + – – – – – + + + + – – + + – + + – – + + – – + – + – – – – – + + + – + + – + – + + – – + + – – + – + – – – – – + + + + – – – – – + + + + – + – + + – – + + – – – + – + + – – + – + – – – – – + + + + – + – + – + – + – + + – – + + – – + – + – – – – – + + + + – + – + – – – – – + + + + – + – + + – – + – + – – + + – – + – + – – – – – + + + + – + – + + – – + – + – – – – – + + + + – + – + + – – + + – – – – + + + + – + – + + – – + + – – + – – – + + + + – + – + + – – + + – – + – + – – – – – + + + – + – + + – – + + – – + – + – – – – + – + + – – + – + – – – – – + + + + – + – + + – – + – + – – – – – + + + + – + – + + – – + + – – + – – + + – – + – + – – – – – + + + + – + + – + – – + – + – – – – – + + + + – + – + + – + – – + + + + – + – + + – – + + – – + – + – – – – – + + – – + + – – + – + – – – – – + + + + + – – + – – – – – + + + + – + – + + – – + + – + – – – + + + + – + – + + – – + + – – + – + – – – – – – + + + + – + – + + – – + + – – + – + – – – – – – – + + + + – + – + + – – + + – – + + P.B.28 + + + + + + + + + + + + + + + + + + + + + + + + + + – + + + – – + + + – + + – – + + + + – – – – – – – – + + + + + – – – – – – – – + + + + – – + + + – + + – – + + + + – + + – – + + + + – – – – – – – – + + + + – – + + + + – + – – + + – – – + – – + + – + – – + – + – + – – + + – + – – + – + – + – + + – + – – + + – – – + – – – + + – – – + – – + + – + – – + – + – + – + + – + – – – + – + + – + – – + + – – – + – – + + – + – – + – + – – + – + – + + – + – – + + – – – + – – + + – + – – + – – + – + – + – + + – + – – + + – – – + – – + + – + – – – + – + – + – + – + – + – + – + – + – + – + – + – + – + + – – + + + + – – – – – – – – + + + + – – + + + – + + + – – + + + – + + – – + + + + – – – – – – – – + + + + + – – – – – – – – + + + + – – + + + – + + – – + + + + – + + + + – – + + + – + + – – + + + + – – – – – – – + – + + + + – – – – – – – – + + + + – – + + + – + + – + – + + + – + + – – + + + + – – – – – – – – + + + + – + – + + – – + + + + – – – – – – – – + + + + – – + + + + – + – – + + – + – – + – + – + – + + – + – – + + – – – – + – – + + – – – + – – + + – + – – + – + – + – + + – – + – – + – + – + – + + – + – – + + – – – + – – + + – – – + + – + – – + – + – + – + + – + – – + + – – – + – – – + + – – – + – – + + – + – – + – + – + – + + – + – – – + – + – + – + + – + – – + + – – – + – – + + – + – – – – + + + + – – + + + – + + – – + + + + – – – – – + – – – + – – + + – + – – + – + – + – + + – + – – + + – – – – – – + + + + – – + + + – + + – – + + + + – – – + – – – – – – – + + + + – – + + + – + + – – + + + + – + P.B.32 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + – – – – – – – – – – – – – – – – + + + + + + + + + + + + + + – + – – – – – – – – + + + + + + + – – – – – – – + + + + + + + – – + + + + + + + + – – – – – – – – – – – – – – + + + + + + – + + – – – – + + + + – – – – + + + – – – – + + + – – – + + + – + – + + + + – – – – + + + + – – – – – – – + + + – – – + + + – – + + + + + – – – – – – – – + + + + + + + – – – – – – + + + – – – – – – – + + + + + + + + – – – + + + + – – – – – – + + – + + + + – – + + – – + + – – + + – – + – – + + – – + – – – + – + + – – + + – – + + – – + + – – + + + – – + + – – + – – – + – + – + – + + – – + + – + – – + + – – – + + – – + + + – – – + – + – – + – – + + – – + – + + – – + + – + + – – + + + – – – + – – + + – + + – + – – + – + + – + – – – + + – + – – – + + – + – – + – + – – + – + + – + – – + – + + – + + – + – – – + + – + – – – + + – – + – + + – – + + – + – – + – – + – + + – + + – + – – – – – + + – + – – + + – – + – + + + – – + – + + – + + – – + + + + – + – + – + – + – + – + – + – – + – + – + – – + – – – + + + – + – + – + – + – + – + – + – + – + – + – + – – + – – – + + – + + – + – + – + – – + – + – + – + – + – + – + – + – – – + + – – – + – + – + – + + – + – + – + + – + – + – + – + – – – + – + + + – + – – + – + + – + – – + – + – + – – + – + – + – – + – + – – + – + + – + – – + – + + – + + – + – – + – + – + – – + – – + – + – + + – + – + – + – – + – – + – + + – + + – + – – + – – – + – + – – + – + – + – + + – + – + – + + – + + – + – – – + + + – – + + – – + + – – + + – – + – – + + – – + – – + + – – + + – + + – – + + – – + + – – + + – – – + + – – + – – + + – – + – + + + – – + + – – – – + + – – + + + – – + + – – – + + – – + – – – – + + – – + + + + – – + + – + + – – + + – – – + + – – – + + + + – – – – + + + + – – – – + + + – – – – + + + – + – – – + – – – + + + + – – – – + + + + – + + – – – – + + + – + – – – – + – – + + + + – – + + – – – – + – – + + + + – + + – + – – – – – + + – – – – + + – – + + + + – – – + + + + – + + – + P.B.36 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + – + + + + – + + + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + + – – – – – – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – + + + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + + + – + – – + – + – + + – – + + – + – – – + – + – – + + – – + – + – – + + – + – – – + – + – – + + – – + – + – + + – + – – + – + – + + – – – + + – – + + – + – – – + – + – – + + – – + – + – + + – + – – + – + – – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – + – + – – – + – + + – + – – + – + – + + – – + + – + – – – + – + – – + + – – + – – + – + + – – + + – + – – – + – + – – + + – – + – + – + + – + – – + – – + – + – + + – + – – + – + – + + – – + + – + – – – + – + – – + + – – – + – + – + + – – + + – + – – – + – + – – + + – – + – + – + + – + – – – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – – – – – – + + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + + – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + – – + + + – – – – – – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – + + + + + – + + + + – – + + – – – – – – + + + + – – – – – – + + – + – + + + + – – + + – – – – – – + + + + – – – – – – + + – – + + + + + + – + + + + – – – – – – + + – – + + + + + – + + + + – – + + – – – – – + – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – – – – – + – + + – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + – + – + – + – – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – – + – – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – + – – + – – + – + – + + – – + + – + – – – + – + – – + + – – + – + – + + – – + – – – + – + – – + + – – + – + – + + – + – – + – + – + + – – + + – – – + + – + – – – + – + – – + + – – + – + – + + – + – – + – + – + + – – – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – + – + – – – + – + – + + – + – – + – + – + + – – + + – + – – – + – + – – + + – – – + – + – + + – – + + – + – – – + – + – – + + – – + – + – + + – + – – – – + + + + – – – – – – + + – – + + + + + – + + + + – – + + – – – + – – – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – – – + – – – + – + – – + + – – + – + – + + – + – – + – + – + + – – + + – + – – – – – – + + + + – – – – – – + + – – + + + + + – + + + + – – + + – + – – – – – + + – – + + + + + – + + + + – – + + – – – – – – + + + + – + P.B.40 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + – + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + – + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + – + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + – + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + – + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + – + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + – + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + – + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – – + + – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + + – + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + + – – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + + – – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + – – + – + + + – – + – + + – – + – + + – + – + – + – – + + – – + + – – + – + – + – + + – + + – – + – + – + – + + – + – – + – + + – – + – + + – + – + – + – – + + – – + + + – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – + + – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – + + + + – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – – – – – – – – – – + + + + – – – – + + – – – – + + + + + + + + – – + + – – + + – – P.B.44 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + – + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – – + + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – – + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + – + + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – – + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – – + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – – + – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + – + – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – – + – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + – + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + – + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + – + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + + – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – + – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + + – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – + – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + + – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – + – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + + – – + – + + + – – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – + – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – – + – – + + – – – + – + – – + + + – + + + + + – – – + – + + + – – – – – + – – – + + – + – + + – + – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – – – – + – – – – + – – – + + – + – + + – – + – – + – + – – + + + – + + + + + – – – + – + + + – P.B.48 + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + – + + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – – + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + – + + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – – + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – – + + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – – + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – – + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + – + – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + – + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + – + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + – + – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – – + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – + – + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + + – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + + – + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – + – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + + – + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + + – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + + – – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + + – – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – + – – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – + – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – + + – + + – – – + – + – + + – – – – + – – – – – + + + + – + + + + – – + – + – + + + – + – – – + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – – + – – – – – – – + + + + – + + + + – – + – + – + + + – – + – – + + – + + – – – + – + – + + – – – +