Extended Exercise: Gaussian Elimination

Mathematicians not only search for solutions of equations in one variable, they also study whole systems of linear equations. Here is a sample system of equations in three variables, x, y, and z:

#displaymath73746#

A solution to a system of equations is a series of numbers, one per variable, such that if we replace the variable by its corresponding number, the two sides of each equation evaluate to the same number. In our running example, the solution is x = 1, y = 1, and z = 2 as we can easily check:

#displaymath73754#

The first equation now reads as 10 = 10, the second one as 31 = 31, and the last one as 1 =1. One of the most famous methods for finding a solution is called Gaussian elimination. It consists of two steps. The first step is to transform the system of equations into a system of different shape but with the same solution. The second step is to find solutions to one equation at a time. Here we focus on the first step because it is another interesting instance of generative recursion. The first step of the Gaussiam elminiation algorithm is called ``triangulation'' because the result is a system of equations in the shape of a triangle. In contrast, the original system is typically a rectangle. To understand this terminology, take a look at this representation of the original system:

#displaymath73762#

This representation captures the essence of the system, namely the numeric coefficients of the variables and the right hand sides. The names of the variables don't play any role. The generative step in the triangulation phase is to subtract the first row (list) of numbers from all the other rows. Subtracting one row from another means subtracting the corresponding items in the two rows. With our running example, this step would yield

#displaymath73764#

when we subtract the first row from the second. The goal of these subtractions is to put a 0 into the first column of all but the first row. To achieve this for the last row, we subtract the first row twice from the second one:

#displaymath73766#

Put differently, we first multiply each item in the first row with <#66753#><#36404#>2<#36404#><#66753#> and then subtract the result from the last row. It is easy to check that the solution for the original system of equations and this new one are identical.
<#36407#>Exercise 27.5.1<#36407#> Check that the following system of equations

#displaymath73768#

has the same solution as the one labeled with #tex2html_wrap_inline73770#. external Solution<#66754#><#66754#> <#36417#>Exercise 27.5.2<#36417#> Develop <#66755#><#36419#>subtract<#36419#><#66755#>. The function consumes two lists of numbers of equal length. It subtracts the first from the second, item by item, enough times so that the result contains <#66756#><#36420#>0<#36420#><#66756#> in the first position. The result is the <#66757#><#36421#>rest<#36421#><#66757#> of this list.~ external Solution<#66758#><#66758#>
Following mathematical convention, we drop the leading <#66759#><#36429#>0<#36429#><#66759#>'s from the representation of the last two equations:

#displaymath73772#

If, in addition, we use the same process for the remainder of the system to generate shorter rows, the final representation has a triangular shape. Let us study this idea with our running example. For the moment we ignore the first row and focus on the rest of the equations:

#displaymath73774#

By subtracting the first row now <#36443#>-1<#36443#> times from the second one, we get

#displaymath73776#

after dropping the leading <#66760#><#36450#>0<#36450#><#66760#>. The remainder of this system is a single equation, which cannot be simplified any further. Here is the result of putting together this last system with the first equation:

#displaymath73778#

As promised, the shape of this system of equations is (roughly) a triangle, and as we can easily check, it has the same solution as the original system.
<#36461#>Exercise 27.5.3<#36461#> Check that the following system of equations

#displaymath73780#

has the same solution as the one labeled with #tex2html_wrap_inline73782#. external Solution<#66761#><#66761#> <#36471#>Exercise 27.5.4<#36471#> Develop the algorithm <#66762#><#36473#>triangulate<#36473#><#66762#>, which consumes a rectangular representation of a system of equations and produces a triangular version according the Gaussian algorithm. external Solution<#66763#><#66763#>
Unfortunately, the current version of the triangulation algorithm occasionally fails to produce the solution. Consider the following (representation of a) system of equations:

#displaymath73784#

Its solution is x = 2, y = 1, and z = 1. The first step is to subtract the first row from the second and to subtract it twice from the last one, which yields the following matrix:

#displaymath73792#

Next our algorithm would focus on the rest of the matrix:

#displaymath73794#

but the first item of this matrix is <#66764#><#36501#>0<#36501#><#66764#>. Since we cannot divide by <#66765#><#36502#>0<#36502#><#66765#>, we are stuck. To overcome this problem, we need to use another piece of knowledge from our problem domain, namely, that we can switch equations around without changing the solution. Of course, as we switch rows, we must make sure that the first item of the row to be moved is not <#66766#><#36503#>0<#36503#><#66766#>. Here we can simply swap the two rows:

#displaymath73796#

From here we may continue as before, subtracting the first equation from the remaining ones a sufficient number of times. The final triangular matrix is:

#displaymath73798#

It is easy to check that this system of equations still has the solution x = 2, y = 1, and z = 1.
<#36519#>Exercise 27.5.5<#36519#> Revise the algorithm <#66767#><#36521#>triangulate<#36521#><#66767#> from exercise~#extriangulate1#36522> so that it switches rows when the first item of the matrix is <#66768#><#36523#>0<#36523#><#66768#>. <#36524#>Hint:<#36524#> DrScheme provides the function <#66769#><#36525#>remove<#36525#><#66769#>. It consumes an item <#66770#><#36526#>I<#36526#><#66770#> and a list <#66771#><#36527#>L<#36527#><#66771#> and produces a list like <#66772#><#36528#>L<#36528#><#66772#> but with the first occurrence of <#66773#><#36529#>I<#36529#><#66773#> removed. For example,

  <#36534#>(remove<#36534#> <#36535#>(list<#36535#> <#36536#>0<#36536#> <#36537#>1)<#36537#> <#36538#>(list<#36538#> <#36539#>(list<#36539#> <#36540#>2<#36540#> <#36541#>1)<#36541#> <#36542#>(list<#36542#> <#36543#>0<#36543#> <#36544#>1)))<#36544#> 
<#36545#>=<#36545#> <#36546#>(list<#36546#> <#36547#>(list<#36547#> <#36548#>2<#36548#> <#36549#>1))<#36549#> 
external Solution<#66774#><#66774#> <#36558#>Exercise 27.5.6<#36558#> Some systems of equations don't have a solution. Consider the following system as an example:

#displaymath73806#

Try to produce a triangular system by hand and with <#66775#><#36563#>triangulate<#36563#><#66775#>. What happens? Modify the function so that it signals an error if it encounters this situation.~ external Solution<#66776#><#66776#> <#36569#>Exercise 27.5.7<#36569#> After we obtain a triangular system of equations such as (*) on page~#pgstarequation#36571> (or exercise~#exgaussequiv2#36572>), we can solve the equations. In our specific example, the last equation says that <#36573#>z<#36573#> is 2. Equipped with this knowledge, we can eliminate <#36574#>z<#36574#> from the second equation through a substitution:

#displaymath73810#

Determine the value for <#36575#>y<#36575#>. Then repeat the substitution step for <#36576#>y<#36576#> and <#36577#>z<#36577#> in the first equation and find the value for <#36578#>x<#36578#>. Develop the function <#66777#><#36579#>solve<#36579#><#66777#>, which consumes triangular systems of equations and produces a solution. A triangular system of equations has the shape

#displaymath73812#

where #tex2html_wrap_inline73814# and #tex2html_wrap_inline73816# are numbers. That is, it is a list of lists and each of the lists is one item shorter than the preceding one. A solution is a list of numbers. The last number on the list is

#displaymath73818#

<#36594#>Hint:<#36594#> Developing <#66779#><#36595#>solve<#36595#><#66779#> requires a solution for the following problem. Suppose we are give a row:

<#36600#>(list<#36600#> <#36601#>3<#36601#> <#36602#>9<#36602#> <#36603#>21)<#36603#>
and a list of numbers that solve the remainder of the system:
<#36611#>(list<#36611#> <#36612#>2).<#36612#>
In the world of equations, these two pieces of data represent the following knowledge:

#displaymath73820#

and

#displaymath73822#

which in turn means we must solve the following equation:

#displaymath73824#

Develop the function <#66780#><#36616#>evaluate<#36616#><#66780#>, which evaluates the rest of the left-hand side of an equation and subtracts the right-hand side from this sum. Equivalently, <#66781#><#36617#>evaluate<#36617#><#66781#> consumes <#66782#><#36618#>(list<#36618#>\ <#36619#>9<#36619#>\ <#36620#>21)<#36620#><#66782#> and <#66783#><#36621#>(list<#36621#>\ <#36622#>2)<#36622#><#66783#> and produces <#66784#><#36623#>-3<#36623#><#66784#>, that is, #tex2html_wrap_inline73826#. Now use <#66785#><#36624#>evaluate<#36624#><#66785#> for the intermediate step in <#66786#><#36625#>solve<#36625#><#66786#>.~<#66787#><#66787#> external Solution