Karmarkar–Karp bin packing algorithms explained

The Karmarkar–Karp (KK) bin packing algorithms are several related approximation algorithm for the bin packing problem.[1] The bin packing problem is a problem of packing items of different sizes into bins of identical capacity, such that the total number of bins is as small as possible. Finding the optimal solution is computationally hard. Karmarkar and Karp devised an algorithm that runs in polynomial time and finds a solution with at most

OPT+l{O}(log2(OPT))

bins, where OPT is the number of bins in the optimal solution. They also devised several other algorithms with slightly different approximation guarantees and run-time bounds.

The KK algorithms were considered a breakthrough in the study of bin packing: the previously-known algorithms found multiplicative approximation, where the number of bins was at most

rOPT+s

for some constants

r>1,s>0

, or at most

(1+\varepsilon)OPT+1

.[2] The KK algorithms were the first ones to attain an additive approximation.

Input

The input to a bin-packing problem is a set of items of different sizes, a1,...an. The following notation is used:

Given an instance I, we denote:

Obviously, FOPT(I) ≤ OPT(I).

High-level scheme

The KK algorithms essentially solve the configuration linear program:

minimize~~1x~~~s.t.~~Ax\geqn~~~and~~x\geq0~~~and~~x~isaninteger~

.
Here, A is a matrix with m rows. Each column of A represents a feasible configuration - a multiset of item-sizes, such that the sum of all these sizes is at most B. The set of configurations is C. x is a vector of size C. Each element xc of x represents the number of times configuration c is used.

There are two main difficulties in solving this problem. First, it is an integer linear program, which is computationally hard to solve. Second, the number of variables is C - the number of configurations, which may be enormous. The KK algorithms cope with these difficulties using several techniques, some of which were already introduced by de-la-Vega and Lueker. Here is a high-level description of the algorithm (where

I

is the original instance):

J

be an instance constructed from

I

by removing small items.

K

be an instance constructed from

J

by grouping items and rounding the size of items in each group to the highest item in the group.

K

, without the integrality constraints.

K

.

J

.

I

.

Below, we describe each of these steps in turn.

Step 1. Removing and adding small items

The motivation for removing small items is that, when all items are large, the number of items in each bin must be small, so the number of possible configurations is (relatively) small. We pick some constant

g\in(0,1)

, and remove from the original instance

I

all items smaller than

gB

. Let

J

be the resulting instance. Note that in

J

, each bin can contain at most

1/g

items. We pack

J

and get a packing with some

bJ

bins.

Now, we add the small items into the existing bins in an arbitrary order, as long as there is room. When there is no more room in the existing bins, we open a new bin (as in next-fit bin packing). Let

bI

be the number of bins in the final packing. Then:

bI\leqmax(bJ,(1+2g)OPT(I)+1)

.
Proof. If no new bins are opened, then the number of bins remains

bJ

. If a new bin is opened, then all bins except maybe the last one contain a total size of at least

B-gB

, so the total instance size is at least

(1-g)B(bI-1)

. Therefore,

FOPT\geq(1-g)(bI-1)

, so the optimal solution needs at least

(1-g)(bI-1)

bins. So

bI\leqOPT/(1-g)+1=(1+g+g2+\ldots)OPT+1\leq(1+2g)OPT+1

.

In particular, by taking g=1/n, we get:

bI\leqmax(bJ,OPT+2OPT(I)/n+1)\leqmax(bJ,OPT+3)

,
since

OPT(I)\leqn

. Therefore, it is common to assume that all items are larger than 1/n.

Step 2. Grouping and un-grouping items

The motivation for grouping items is to reduce the number of different item sizes, to reduce the number of constraints in the configuration LP. The general grouping process is:

There are several different grouping methods.

Linear grouping

Let

k>1

be an integer parameter. Put the largest

k

items in group 1; the next-largest

k

items in group 2; and so on (the last group might have fewer than

k

items). Let

J

be the original instance. Let

K'

be the first group (the group of the

k

largest items), and

K

the grouped instance
without the first group. Then:

K'

all items have the same size. In

K

the number of different sizes is

m(K)\leqn/k+1

.

OPT(K)\leqOPT(J)

- since group 1 in

J

dominates group 2 in

K

(all k items in group 1 are larger than the k items in group 2); similarly, group 2 in

J

dominates group 3 in

K

, etc.

OPT(K')\leqk

- since it is possible to pack each item in

K'

into a single bin.

Therefore,

OPT(J)\leqOPT(K\cupK')\leqOPT(K)+OPT(K')\leqOPT(K)+k

. Indeed, given a solution to

K

with

bK

bins, we can get a solution to

J

with at most

bK+k

bins.

Geometric grouping

Let

k>1

be an integer parameter. Geometric grouping proceeds in two steps:

J

into several instances

J0,J1,\ldots

such that, in each instance

Jr

, all sizes are in the interval

[B/2r+1,B/2r)

. Note that, if all items in

J

have size at least

gB

, then the number of instances is at most

log2(1/g)

.

Jr

, perform linear rounding with parameter

k2r

. Let

Kr,K'r

be the resulting instances. Let

K:=\cuprKr

and

K':=\cuprK'r

.Then, the number of different sizes is bounded as follows:

m(K'r)=1

and

m(Kr)\leqn(Jr)/(k2r)+1

. Since all items in

Jr

are larger than

B/2r+1

, we have

n(Jr)\leq2r+1FOPT(Jr)

, so

m(Kr)\leq2FOPT(Jr)/k+1

. Summing over all r gives

m(K)\leq2FOPT(J)/k+log2(1/g)

.

The number of bins is bounded as follows:

OPT(K'r)\leqk

- since

K'r

has

k2r

items, and all of them are smaller than

B/2r

, so they can be packed into at most

k

bins.

OPT(K')\leqklog2(1/g)

.

OPT(J)\leqOPT(K)+OPT(K')\leqOPT(K)+klog2(1/g)

.

Alternative geometric grouping

Let

k>1

be an integer parameter. Order the items by descending size. Partition them into groups such that the total size in each group is at least

kB

. Since the size of each item is less than B, The number of items in each group is at least

k+1

. The number of items in each group is weakly-increasing. If all items are larger than

gB

, then the number of items in each group is at most

k/g

. In each group, only the larger items are rounded up. This can be done such that:

m(K)\leqFOPT(J)/k+ln(1/g)

.

OPT(J)\leqOPT(K)+2k(2+ln(1/g))

.

Step 3. Constructing the LP and rounding the solution

We consider the configuration linear program without the integrality constraints:

minimize~~1x~~~s.t.~~Ax\geqn~~~and~~x\geq0

.
Here, we are allowed to use a fractional number of each configuration.

Denote the optimal solution of the linear program by LOPT. The following relations are obvious:

A solution to the fractional LP can be rounded to an integral solution as follows.

Suppose we have a solution x to the fractional LP. We round x into a solution for the integral ILP as follows.

\sumc\inxc=bL

bins (note that

bL

may be a fractional number). Since the fractional LP has m constraints (one for each distinct size), x has at most m nonzero variables, that is, at most m different configurations are used. We construct from x an integral packing consisting of a principal part and a residual part.

bL+m/2

bins.

This also implies that

OPT(I)\leqLOPT(I)+m/2

.

Step 4. Solving the fractional LP

The main challenge in solving the fractional LP is that it may have a huge number of variables - a variable for each possible configuration.

The dual LP

The dual linear program of the fractional LP is:

maximize~~ny~~~s.t.~~ATy\leq1~~~and~~y\geq0

.
It has m variables

y1,\ldots,ym

, and C constraints - a constraint for each configuration. It has the following economic interpretation. For each size s, we should determine a nonnegative price

yi

. Our profit is the total price of all items. We want to maximize the profit n y subject to the constraints that the total price of items in each configuration is at most 1. This LP now has only m variables, but a huge number of constraints. Even listing all the constraints is infeasible.

Fortunately, it is possible to solve the problem up to any given precision without listing all the constraints, by using a variant of the ellipsoid method. This variant gets as input, a separation oracle: a function that, given a vector y ≥ 0, returns one of the following two options:

ATy\leq1

; or -

ay>1

.The ellipsoid method starts with a large ellipsoid, that contains the entire feasible domain

ATy\leq1

. At each step t, it takes the center

yt

of the current ellipsoid, and sends it to the separation oracle:

yt

is feasible, then we do an "optimality cut": we cut from the ellipsoid all points y for which

ny<nyt

. These points are definitely not optimal.

yt

is infeasible and violates the constraint a, then we do a "feasibility cut": we cut from the ellipsoid all points y for which

ay>1

. These points are definitely not feasible.

After making a cut, we construct a new, smaller ellipsoid. It can be shown that this process converges to an approximate solution, in time polynomial in the required accuracy.

A separation oracle for the dual LP

We are given some m non-negative numbers

y1,\ldots,ym

. We have to decide between the following two options:

yi

corresponding to this configuration is at most 1; this means that y is feasible.

yi

is larger than 1; this means that y is infeasible. In this case, we also have to return the configuration.

This problem can be solved by solving a knapsack problem, where the item values are

y1,\ldots,ym

, the item weights are

s1,\ldots,sm

, and the weight capacity is B (the bin size).

ay>1

for the vector a that corresponds to this configuration).The knapsack problem can be solved by dynamic programming in pseudo-polynomial time:

O(mV)

, where m is the number of inputs and V is the number of different possible values. To get a polynomial-time algorithm, we can solve the knapsack problem approximately, using input rounding. Suppose we want a solution with tolerance

\delta

. We can round each of

y1,\ldots,ym

down to the nearest multiple of

\delta

/n. Then, the number of possible values between 0 and 1 is n/

\delta

, and the run-time is

O(mn/\delta)

. The solution is at least the optimal solution minus

\delta

/n.

Ellipsoid method with an approximate separation oracle

The ellipsoid method should be adapted to use an approximate separation oracle. Given the current ellipsoid center

yt

:

yt

is definitely infeasible, and the solution correspond to a configuration that violates a constraint a. We do a "feasibility cut" in

yt

, cutting the ellipsoid all points y for which

ay>1

.

yt

may or may not be feasible, but

yt

rounded down (denote it by

zt

) is feasible. By definition of the rounding, we know that

nzt\geqnyt-n1(\delta/n)=nyt-\delta

. We still do an "optimality cut" in

yt

: we cut from the ellipsoid all points y for which

ny<nyt

. Note that

yt

might be infeasible, so its value might be larger than OPT. Therefore, we might remove some points whose objective is optimal. However, the removed points satisfy

ny<nzt+\delta\leqOPT+\delta

; no point is removed if its value exceeds the value at

zt

by more than

\delta

.Using the approximate separation oracle gives a feasible solution
y* to the dual LP, with

n

y*

\geqLOPT-\delta

, after at most

Q

iterations, where

Q=4m2ln(mn/g\delta)

. The total run-time of the ellipsoid method with the approximate separation oracle is

O(Qmn/\delta)

.

Eliminating constraints

During the ellipsoid method, we use at most Q constraints of the form

ay\leq1

. All the other constraints can be eliminated, since they have no effect on the outcome y* of the ellipsoid method. We can eliminate even more constraints. It is known that, in any LP with m variables, there is a set of m constraints that is sufficient for determining the optimal solution (that is, the optimal value is the same even if only these m constraints are used). We can repeatedly run the ellipsoid method as above, each time trying to remove a specific set of constraints. If the resulting error is at most

\delta

, then we remove these constraints permanently. It can be shown that we need at most

(Q/m)+mln(Q/m)

eliminations, so the accumulating error is at most

\delta[(Q/m)+mln(Q/m)]

. If we try sets of constraints deterministically, then in the worst case, one out of m trials succeeds, so we need to run the ellipsoid method at most

m[(Q/m)+mln(Q/m)] =Q+m2ln(Q/m)

times. If we choose the constraints to remove at random, then the expected number of iterations is

O(m)[1+ln(Q/m)]

.

Finally, we have a reduced dual LP, with only m variables and m constraints. The optimal value of the reduced LP is at least

LOPT-h

, where

h\delta[(Q/m)+mln(Q/m)]

.

Solving the primal LP

By the LP duality theorem, the minimum value of the primal LP equals the maximum value of the dual LP, which we denoted by LOPT. Once we have a reduced dual LP, we take its dual, and take a reduced primal LP. This LP has only m variables - corresponding to only m out of C configurations. The maximum value of the reduced dual LP is at least

LOPT-h

. It can be shown that the optimal solution of the reduced primal LP is at most

LOPT+h

. The solution gives a near-optimal bin packing, using at most m configurations.

The total run-time of the deterministic algorithm, when all items are larger than

gB

, is:
O\left(Qmn
\delta

(

2lnQ
m
Q+m) \right) = O \left(
2
Qmn+Qm3
nlnQ
m
\delta

\right) O\left(m8ln{m}

2(mn
gh
ln

)+

m4nln{m
}\ln(\frac) \right),

The expected total run-time of the randomized algorithm is:

O\left(m7log{m}

2(mn
gh
log

)+

m4nlog{m
}\log(\frac) \right).

End-to-end algorithms

Karmarkar and Karp presented three algorithms, that use the above techniques with different parameters. The run-time of all these algorithms depends on a function

T(,)

, which is a polynomial function describing the time it takes to solve the fractional LP with tolerance h=1, which is, for the deterministic version,

T(m,n)\inO(m8log{m}log2{n}+m4nlog{m}log{n})

.

Algorithm 1

Let

\epsilon>0

be a constant representing the desired approximation accuracy.

g=max(1/n,\epsilon/2)

. Let

J

be an instance constructed from

I

by removing all items smaller than g.

k=n\epsilon2

. Let

K

be an instance constructed from

J

by linear grouping with parameter k, and let

K'

be the remaining instance (the group of k largest items). Note that

m(K)\leqn/k+11/\epsilon2

.

K

, without the integrality constraints.

K

, with tolerance h=1. The result is a fractional bin packing with

bL\leqLOPT(K)+1

bins. The run-time is

T(m(K),n(K))\leqT(\epsilon-2,n)

.

K

. Add at most

m(K)/2

bins for the fractional part. The total number of bins is

bK\leqbL+m(K)/2\leqLOPT(K)+1+1/2\epsilon2

.

K'

using at most k bins; get a packing of

J

. The number of bins is

bJ\leqbK+k\leqLOPT(K)+1+1/2\epsilon2+n\epsilon2

.

I

. The number of bins is:

bI\leqmax(bJ,(1+2g)OPT(I)+1)\leq(1+\epsilon)OPT(I)+1/2\epsilon2+3

.

All in all, the number of bins is in

(1+\epsilon)OPT+O(\epsilon-2)

and the run-time is in

O(nlogn+T(\epsilon-2,n))

. By choosing

\epsilon=OPT-1/3

we get

OPT+O(OPT2/3)

.

Algorithm 2

Let

g>0

be a real parameter and

k>0

an integer parameter to be determined later.

J

be an instance constructed from

I

by removing all items smaller than g.

FOPT(J)>1+

k
k-1

ln(1/g)

do:

K

be the resulting instance, and let

K'

be the remaining instance. We have

m(K)\leqFOPT(J)/k+ln(1/g)

.

K

, without the integrality constraints.

K

, with tolerance h=1. The result is a fractional bin packing with

bL\leqLOPT(K)+1

bins. The run-time is

T(m(K),n(K))\leqT(FOPT(J)/k+ln(1/g),n)

.

K

. Do not add bins for the fractional part. Instead, just remove the packed items from

J

.

K'

in at most

2k(2+ln(1/g))

bins.

FOPT(J)\leq1+

k
k-1

ln(1/g)

, pack the remaining items greedily into at most

2FOPT(J)\leq2+

2k
k-1

ln(1/g)

bins.

FOPT(Jt+1)\leqm(Kt)\leqFOPT(Jt)/k+ln(1/g)

. The FOPT drops by a factor of k in each iteration, so the number of iterations is at most
lnFOPT(I)
lnk

+1

.

J

is:

bJ\leqOPT(I)+\left[1+

lnFOPT(I)
lnk

\right]\left[1+4k+2kln(1/g)\right]+2+

2k
k-1

ln(1/g)

.

I

. The number of bins is:

bI\leqmax(bJ,(1+2g)OPT(I)+1)

.

The run-time is in

O(nlogn+T(FOPT(J)/k+ln(1/g),n))

.

Now, if we choose k=2 and g=1/FOPT(I), we get:

bJ\leqOPT+O(log2(FOPT))

,
and hence:

bI\leqmax(bJ,OPT+2OPT/FOPT+1)\leqmax(bJ,OPT+5)\inOPT+log2(OPT)

,
so the total number of bins is in

OPT+O(log2(FOPT))

. The run-time is

O(nlogn)+T(FOPT/2+ln(FOPT),n)\inO(nlog{n}+T(FOPT,n))

.

The same algorithm can be used with different parameters to trade-off run-time with accuracy. For some parameter

\alpha\in(0,1)

, choose

k=FOPT\alpha

and

g=1/FOPT1-\alpha

. Then, the packing needs at most

OPT+l{O}(OPT\alpha)

bins, and the run-time is in

O(nlog{n}+T(FOPT(1-\alpha),n))

.

Algorithm 3

The third algorithm is useful when the number of sizes m is small (see also high-multiplicity bin packing).

g=

log2(m)
FOPT(I)
. Let

K

be an instance constructed from

I

by removing all items smaller than g.

m(K)\leqFOPT(K)

then:

K

, without the integrality constraints.

K

, with tolerance h=1. The result is a fractional bin packing with

bL\leqLOPT(K)+1

bins. The run-time is

T(m(K),n(K))\leqT(\epsilon-2,n)

.

K

. Do not add bins for the fractional part. Instead, just remove the packed items from

K

.

I

. The number of bins is:

bI\leqmax(bJ,(1+2g)OPT(I)+1)

.

It uses at most

OPT+l{O}(log2m)

bins, and the run-time is in

O(nlog{n}+T(m,n))

.

Improvements

The KK techniques were improved later, to provide even better approximations.

Rothvoss[4] uses the same scheme as Algorithm 2, but with a different rounding procedure in Step 2. He introduced a "gluing" step, in which small items are glued together to yield a single larger item. This gluing can be used to increase the smallest item size to about

B/log12(n)

. When all sizes are at least

B/log12(n)

, we can substitute

g=1/log12(n)

in the guarantee of Algorithm 2, and get:

bJ\leqOPT(I)+O(log(FOPT)log(log(n)))

,
which yields a

OPT+O(log(OPT)loglog(OPT))

bins.

Hoberg and Rothvoss use a similar scheme in which the items are first packed into "containers", and then the containers are packed into bins. Their algorithm needs at most

bJ\leqOPT(I)+O(log(OPT))

bins.

Notes and References

  1. Karmarkar. Narendra. Karp. Richard M.. November 1982. An efficient approximation scheme for the one-dimensional bin-packing problem. 23rd Annual Symposium on Foundations of Computer Science (SFCS 1982). 312–320. 10.1109/SFCS.1982.61. 18583908.
  2. Fernandez de la Vega. W.. Lueker. G. S.. 1981. Bin packing can be solved within 1 + ε in linear time. Combinatorica. en. 1. 4. 349–355. 10.1007/BF02579456. 1439-6912. 10519631.
  3. Web site: Claire Mathieu. Approximation Algorithms Part I, Week 3: bin packing. Coursera.
  4. Book: Rothvoß, T.. 2013 IEEE 54th Annual Symposium on Foundations of Computer Science . Approximating Bin Packing within O(log OPT · Log Log OPT) Bins . 2013-10-01. https://ieeexplore.ieee.org/document/6686137. 20–29. 1301.4010. 10.1109/FOCS.2013.11. 978-0-7695-5135-7. 15905063.