It is time to introduce a rigorous description of the phrase ``on the order
of'' and to explain why it is acceptable to ignore some constants. Any
serious programmer must be thoroughly familiar with this notion. It is the
most fundamental method for analyzing and comparing the behavior of
programs. This intermezzo provides a first glimpse at the idea; a second
course on computing usually provides some more in-depth considerations.
#picture38175#
<#38186#>Figure: A comparions of two running time expressions<#38186#>
Let's consider a sample ``order of'' claim with concrete examples before we
agree on a definition. Recall that a function <#67016#><#38188#>F<#38188#><#67016#> may require on the
order of <#38189#>N<#38189#> steps and a function <#67017#><#38190#>G<#38190#><#67017#> #tex2html_wrap_inline73858# steps, even though
both compute the same results for the same inputs. Now suppose the basic
time constants are <#67018#><#38191#>1000<#38191#><#67018#> for <#67019#><#38192#>F<#38192#><#67019#> and 1 for <#67020#><#38193#>G<#38193#><#67020#>. One
way to compare the two claims is to tabulate the abstract running time:
#tabular38195#
At first glance the table seems to say that <#67023#><#38202#>G<#38202#><#67023#>'s performance is
better than <#67024#><#38203#>F<#38203#><#67024#>'s, because for inputs of the same size (<#38204#>N<#38204#>),
<#67025#><#38205#>G<#38205#><#67025#>'s running time is always smaller than <#67026#><#38206#>F<#38206#><#67026#>'s. But a
closer look reveals that as the inputs get larger, <#67027#><#38207#>G<#38207#><#67027#>'s advantage
decreases. Indeed, for an input of size 1000, the two functions need the
same number of steps, and thereafter <#67028#><#38208#>G<#38208#><#67028#> is always slower than
<#67029#><#38209#>F<#38209#><#67029#>. Figure~#figbigoh#38210> compares the graphs of the two
expressions. It shows that the linear graph for #tex2html_wrap_inline73864# dominates
the curve of #tex2html_wrap_inline73866# for some finite number of points but thereafter it
is below the curve.
The concrete example recalls two important facts about our informal
discussion of abstract running time. First, our abstract description of
running time is always a claim about the relationship between two
quantities: the size of the input and the number of natural recursions
evaluated. More precisely, the relationship is a (mathematical) function
that maps an abstract size measure of the input to an abstract measure of
the running time. Second, when we compare ``on the order of'' properties of
functions, such as
#displaymath73868#
we really mean to compare the corresponding functions that consume <#38212#>N<#38212#> and produce the above results. In short, a statement concerning the
order of things compares two functions on natural numbers
(<#67030#><#38213#>N<#38213#><#67030#>).
The comparison of functions on <#67031#><#38214#>N<#38214#><#67031#> is difficult because
they are infinite. If a function <#38215#>f<#38215#> produces larger outputs than
some other function <#38216#>g<#38216#> for all natural numbers, then <#38217#>f<#38217#> is
clearly larger than <#38218#>g<#38218#>. But what if this comparison fails for just a
few inputs? Or for a 1,000 such as the one illustrated in
figure~#figbigoh#38219>? Because we would still like to make approximate
judgements, programmers and scientists adapt the mathematical notion of
comparing functions up to some factor and some finite number of exceptions.
<#67032#><#38221#>ORDER-OF (BIG-O)<#38221#><#67032#>: Given a function g on the natural numbers,
O(g) (pronounced: ``on the order of g'' or ``big-O of g'') is a class
of functions on natural numbers. A function <#38222#>f<#38222#> is in O(g) if
there exist numbers <#38223#>c<#38223#> and <#38224#>bigEnough<#38224#> such that for all #tex2html_wrap_inline73880#, it is true that #tex2html_wrap_inline73882#
Recall the performance of <#38227#>F<#38227#> and <#38228#>G<#38228#> from above. For the
first, we assumed that it consumed time according to the following function
#displaymath73884#
the performance of second one obeyed the function <#38229#>g<#38229#>:
#displaymath73886#
Using the definition of big-O, we can say that <#38230#>f<#38230#> is O(g), because
for all #tex2html_wrap_inline73890#,
#displaymath73892#
which means #tex2html_wrap_inline73894# and c = 1.
More importantly, the definition of big-O provides us with a shorthand for
stating claims about a functions running time. For example, from now on, we
say <#67033#><#38232#>how-many<#38232#><#67033#>'s running time is O(N). Keep in mind that <#38233#>N<#38233#>
is the standard abbreviation of the (mathematical) function g(N) =
N. Similarly, we can say that, in the worst case, <#67034#><#38234#>sort<#38234#><#67034#>'s running
time is #tex2html_wrap_inline73902# and <#67035#><#38235#>max<#38235#><#67035#>'s is #tex2html_wrap_inline73904#.
Finally, the definition of big-O explains why we don't have to pay
attention to specific constants in our comparsions of abstract running
time. Consider <#67036#><#38236#>max<#38236#><#67036#> and <#67037#><#38237#>max2<#38237#><#67037#>. We know that <#67038#><#38238#>max<#38238#><#67038#>'s
worst-case running time is in #tex2html_wrap_inline73906#, <#67039#><#38239#>max2<#38239#><#67039#>'s is in O(N). Say,
we need the maximum of a list with 10 numbers. Assuming <#67040#><#38240#>max<#38240#><#67040#> and
<#67041#><#38241#>max2<#38241#><#67041#> roughly consume the same amount of time per basic step,
<#67042#><#38242#>max<#38242#><#67042#> will need #tex2html_wrap_inline73910# steps and <#67043#><#38244#>max2<#38244#><#67043#> will need 10
steps, which means <#67044#><#38245#>max2<#38245#><#67044#> will be faster. Now even if
<#67045#><#38246#>max2<#38246#><#67045#>'s basic step requires twice as much time as <#67046#><#38247#>max<#38247#><#67046#>'s
basic step, <#67047#><#38248#>max2<#38248#><#67047#> is still around 50 times faster. Futhermore, if
we double the size of the input list, <#67048#><#38249#>max<#38249#><#67048#>'s apparent disadvantage
totally disappears. In general, the larger the input is, the less relevant
are the specific constants.
<#38252#>Exercise 29.2.1<#38252#>
In the first subsection, we stated that the function #tex2html_wrap_inline73912#
belongs to the class #tex2html_wrap_inline73914#. Determine the pair of numbers c and <#38254#>bigEnough<#38254#> that verify this claim.~ 
Solution<#67049#><#67049#>
<#38260#>Exercise 29.2.2<#38260#>
Consider the functions #tex2html_wrap_inline73918# and #tex2html_wrap_inline73920#. Show that
<#38262#>g<#38262#> belongs to O(f), which means that <#38263#>f<#38263#> is abstractly
speaking more (or at least equally) expensive than <#38264#>g<#38264#>. If the input
size is guaranteed to be between 3 and 12, which function is
better?~ Solution<#67050#><#67050#>
<#38270#>Exercise 29.2.3<#38270#>
Compare #tex2html_wrap_inline73924# and #tex2html_wrap_inline73926#. Does f belong to O(g)
and/or g to O(f)?~ Solution<#67051#><#67051#>