zotero-db/storage/DAM77ELQ/.zotero-ft-cache

The Linear Algebra Survival Guide: Illustrated with Mathematica

The Linear Algebra
Survival Guide
Illustrated with Mathematica
Fred E. Szabo, PhD Concordia University
Montreal, Canada
AMSTERDAM • BOSTON • HEIDELBERG • LONDON • NEW YORK • OXFORD PARIS • SAN DIEGO • SAN FRANCISCO • SINGAPORE • SYDNEY • TOKYO
Academic Press is an Imprint of Elsevier

Academic Press is an imprint of Elsevier 125, London Wall, EC2Y 5AS. 525 B Street, Suite 1800, San Diego, CA 92101-4495, USA 225 Wyman Street, Waltham, MA 02451, USA The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, UK
Copyright © 2015 Elsevier Inc. All rights reserved.
No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangement with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions
This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).
Notices Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.
Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.
To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.
ISBN: 978-0-12-409520-5
British Library Cataloguing in Publication Data A catalogue record for this book is available from the British Library
Library of Congress Cataloging-in-Publication Data A catalog record for this book is available from the Library of Congress
For information on all Academic Press visit our website at http://store.elsevier.com/
Printed and bound in the USA

1

2

3

4

5

6

7

8

9

1

2

3

4

5

6

7

8

9

About the Matrix Plot
The image on the previous page is a Mathematica matrix plot of a random 9-by-9 matrix with integer elements between -9 and 9.
Random matrices are used throughout the book where matrix forms are required to illustrate concepts, properties, or calculations, but where the numerical content of the illustrations is largely irrelevant.
The presented image shows how matrix forms can be visualized as two-dimensional blocks of color or shades of gray.
MatrixForm[A = RandomInteger[{-− 9, 9}, {9, 9}]]
-− 4 -− 9 1 6 8 5 6 7 -− 8 1 -− 5 -− 5 2 -− 8 8 8 1 -− 8 -− 7 -− 4 2 -− 1 9 3 1 7 6 -− 3 -− 1 6 4 -− 9 1 -− 2 0 9 7 -− 8 -− 4 -− 1 -− 6 -− 8 5 5 1 7 -− 5 -− 3 -− 3 -− 1 -− 2 -− 9 8 -− 1 -− 1 6 5 6 -− 5 4 7 -− 9 5 -− 7 8 -− 7 5 4 -− 1 1 -− 5 4 8 -− 2 8 7 -− 8 -− 9 4 -− 3 -− 7
MatrixPlot[A]

Preface
The principal goal in the preparation of this guide has been to make the book useful for students, teachers, and researchers using linear algebra in their work, as well as to make the book sufficiently complete to be a valuable reference source for anyone needing to understand the computational aspects of linear algebra or intending to use Mathematica to extend their knowledge and understanding of special topics in mathematics.
This book is both a survey of basic concepts and constructions in linear algebra and an introduction to the use of Mathematica to represent them and calculate with them. Some familiarity with Mathematica is therefore assumed. The topics covered stretch from adjacency matrices to augmented matrices, back substitution to bilinear functionals, Cartesian products of vector spaces to cross products, defective matrices to dual spaces, eigenspaces to exponential forms of complex numbers, finite-dimensional vector spaces to the fundamental theorem of algebra, Gaussian elimination to Gram–Schmidt orthogonalization, Hankel matrices to Householder matrices, identity matrices to isomorphisms of vector spaces, Jacobian determinants to Jordan matrices, kernels of linear transformations to Kronecker products, the law of cosines to LU decompositions, Manhattan distances to minimal polynomials, vector and matrix norms to the nullity of matrices, orthogonal complements to overdetermined linear systems, Pauli spin matrices to the Pythagorean theorem, QR decompositions to quintic polynomials, random matrices to row vectors, scalars to symmetric matrices, Toeplitz matrices to triangular matrices, underdetermined linear systems to uppertriangular matrices, Vandermonde matrices to volumes of parallelepipeds, well-conditioned matrices to Wronskians, and zero matrices to zero vectors.
All illustrations in the book can be replicated and used to discover the beauty and power of Mathematica as a platform for a new kind of learning and understanding. The consistency and predictability of the Wolfram Language on which Mathematica is built are making it much easier to concentrate on the mathematics rather than on the computer code and programming features required to produce correct, understandable, and often inspiring mathematical results. In addition, the included manipulations of many of the mathematical examples in the book make it easy and instructive to explore mathematical concepts and results from a computational point of view.
The book is based on my lecture notes, written over a number of years for several undergraduate and postgraduate courses taught with various iterations of Mathematica. I hereby thank the hundreds of students who have patiently sat through interactive Mathematica-based lectures and have enjoyed the speculative explorations of a large variety of mathematical topics which only the teaching and learning with Mathematica makes possible. The guide also updates the material in the successful textbook “Linear Algebra: An Introduction Using Mathematica,” published by Harcourt/Academic Press over a decade ago.
The idea for the format of this book arose in discussion with Patricia Osborn, my editor at Elsevier/Academic Press at the time. It is based on an analysis of what kind of guide could be written that meets two objectives: to produce a comprehensive reference source for the conceptual side of linear algebra and, at the same time, to provide the reader with the computational illustrations required to learn, teach, and use linear algebra with the help of Mathematica. I am grateful to the staff at Elsevier/Academic Press, especially Katey Birtcher, Sarah Watson and Cathleen Sether for seeing this project through to its successful conclusion and providing tangible support for the preparation of the final version of the book. Last but not least I would like to thank Mohanapriyan Rajendran (Project Manager S&T, Elsevier, Chennai) for his delightful and constructive collaboration during the technical stages of the final composition and production.

xii | The Linear Algebra Survival Guide
Many students and colleagues have helped shape the book. Special thanks are due to Carol Beddard and David Pearce, two of my teaching and research assistants. Both have helped me focus on user needs rather than excursions into interesting but esoteric topics. Thank you Carol and David. Working with you was fun and rewarding. I am especially thankful to Stephen Wolfram for his belief in the accessibility of the computable universe provided that we have the right tools. The evolution and power of the Wolfram Language and Mathematica have shown that they are the tools that make it all possible.
Fred E Szabo Beaconsfield, Quebec
Fall 2014

Dedication
To my family: Isabel, Julie and Stuart, Jahna and Scott, and Jessica, Matthew, Olivia, and Sophie

About the Author
Fred E. Szabo Department of Mathematics, Concordia University, Montreal, Quebec, Canada
Fred E. Szabo completed his undergraduate studies at Oxford University under the guidance of Sir Michael Dummett, and received a Ph.D. in mathematics from McGill University under the supervision of Joachim Lambek. After postdoctoral studies at Oxford University and visiting professorships at several European universities, he returned to Concordia University as a faculty member and dean of graduate studies. For more than twenty years, he developed methods for the teaching of mathematics with technology. In 2012 he was honored at the annual Wolfram Technology Conference for his work on "A New Kind of Learning" with a Wolfram Innovator Award. He is currently professor and Provost Fellow at Concordia University.
Professor Szabo is the author of five Academic Press publications: - The Linear Algebra Survival Guide, 1st Edition - Actuaries' Survival Guide, 1st Edition - Actuaries' Survival Guide, 2nd Edition - Linear Algebra: Introduction Using Maple, 1st Edition - Linear Algebra: Introduction Using Mathematica, 1st Edition

Introduction
How to use this book
This guide is meant as a standard reference to definitions, examples, and Mathematica techniques for linear algebra. Complementary material can be found in the Help sections of Mathematica and on the Wolfram Alpha website. The main purpose of the guide is therefore to collect, in one place, the fundamental concepts of finite-dimensional linear algebra and illustrate them with Mathematica. The guide contains no proofs, and general definitions and examples are usually illustrated in two, three, and four dimensions, if there is no loss of generality. The organization of the material follows both a conceptual and an alphabetic path, whichever is most appropriate for the flow of ideas and the coherence of the presentation. All linear algebra concepts covered in this book are explained and illustrated with Mathematica calculations, examples, and additional manipulations. The Mathematica code used is complete and can serve as a basis for further exploration and study. Examples of interactive illustrations of linear algebra concepts using the Manipulate command of Mathematica are included in various sections of the guide to show how the illustrations can be used to explore computational aspects of linear algebra.
Linear algebra
From a computational point of view, linear algebra is the study of algebraic linearity, the representation of linear transformations by matrices, the axiomatization of inner products using bilinear forms, the definition and use of determinants, and the exploration of linear systems, augmented matrices, matrix equations, eigenvalues and eigenvectors, vector and matrix norms, and other kinds of transformations, among them affine transformations and self-adjoint transformations on inner product spaces. In this approach, the building blocks of linear algebra are systems of linear equations, real and complex scalars, and vectors and matrices. Their basic relationships are linear combinations, linear dependence and independence, and orthogonality. Mathematica provides comprehensive tools for studying linear algebra from this point of view.
Mathematica
The building blocks of this book are scalars (real and complex numbers), vectors, linear equations, and matrices. Most of the time, the scalars used are integers, playing the notationally simpler role of real numbers. In some places, however, real numbers as decimal expansions are needed. Since real numbers may require infinite decimal expansions, both recurring and nonrecurring, Mathematica can represent them either symbolically, such as ⅇ and π𝜋, or as decimal approximations. By default, Mathematica works to 19 places to the right of the decimal point. If greater accuracy is required, default settings can be changed to accommodate specific computational needs. However, questions of computational accuracy play a minor role in this book. In this guide, we follow the lead of Mathematica and avoid the use of ellipses (lists of dots such as "...") to make general statements. In practically all cases, the statements can be illustrated with examples in two, three, and four dimensions. We can therefore also avoid the use of sigmas (Σ) to express sums. The book is written with and for Mathematica 10. However, most illustrations are backward compatible with earlier versions of Mathematica or have equivalent representations. In addition, the natural language interface and internal link to Wolfram/Alpha extends the range of topics accessible through this guide.
Mathematica cells
Mathematica documents are called notebooks and consist of a column of subdivisions called cells. The properties of notebooks and cells are governed by stylesheets. These can be modified globally in the Mathematica Preferences or cellby-cell, as needed. The available cell types in a document are revealed by activating the toolbars in the Window > Show Toolbar menu. Unless Mathematica is used exclusively for input–output calculations, it is advisable to show the toolbar immediately after creating a notebook or to make Show Toolbar a default notebook setting.

2 | The Linear Algebra Survival Guide
Mathematica documents are called notebooks and consist of a column of subdivisions called cells. The properties of notebooks and cells are governed by stylesheets. These can be modified globally in the Mathematica Preferences or cellby-cell, as needed. The available cell types in a document are revealed by activating the toolbars in the Window > Show Toolbar menu. Unless Mathematica is used exclusively for input–output calculations, it is advisable to show the toolbar immediately after creating a notebook or to make Show Toolbar a default notebook setting.
Mathematica documentation
Mathematica Help is extensive and systematic. To look for help, the Help > Documentation Center command will produce access to the search field of the Documentation Center.
Quitting the Mathematica kernel
It sometimes happens that we would like to abort a calculation or other Mathematica activity. The command Quit aborts the current computation and annuls all computed and assigned values to variables and other objects. Selecting Evaluation > Quit Kernel > Local is equivalent to invoking the Quit command.
Clearing assigned and computed values
The commands Clear and ClearAll can be used to remove previously assigned values to specific variables and symbols without resetting other definitions and assignments to their default. For example, typing Clear[x,y] into an input cell will remove values previously assigned to the variables x and y.
Generalizing illustrations with Manipulations
Many entries of the guide contain interactive manipulation sections. The manipulations can be used to explore the effect of numerical input changes on outputs. In particular, they provide a setting for “what if?” type questions. The ranges of the Manipulate parameters are usually arbitrarily chosen. They are easily modified to explore specific numerical questions. The Manipulate feature of Mathematica is explained and documented in the Wolfram Documentation section of Mathematica.
Predictive interface
Starting with Mathematica 9, the writing of Mathematica commands has become amazingly simple. A Suggestion Bar usually appears in any new input cell that is alphabetically organized and tries to anticipate both the built-in and userdefined concepts and definitions the user is about to type. This feature is amazing. It not only facilitates the writing of Mathematica code but also avoids having to respect specific grammatical conventions and spellings. The Suggestion Bar contains other features that make working with Mathematica a joy and much less code-dependent.
Assumptions about prior knowledge
This guide focuses on the learning, teaching, and review of linear algebra. Mathematica is the principal tool for doing so. It is therefore assumed that the reader has a basic knowledge of Mathematica. However, the illustrations can be followed, modified, and extended by mimicking the given examples.
The Wolfram Language
All Mathematica commands begin with capital letters. It is therefore advisable to use lower case letters to name defined objects. When naming matrices, this recommendation is usually not followed in this guide in order to make the presentation conform to the usual notations of linear algebra. Vectors and matrices are often presented in row form as lists and lists of lists, surrounded by curly brackets ({}). Twodimensional displays of vectors and matrices can be built by using Mathematica palettes. Command + Enter and Command + Comma in OS X, or Control + Enter and Control + Comma in Windows add columns and rows to the palettes for larger vectors and matrices. For easier readability or conceptual visualization, two-dimensional outputs of matrices are always forced by embedding the specification of the matrices in the MatrixForm commands.

The Linear Algebra Survival Guide | 3
dimensional displays of vectors and matrices can be built by using Mathematica palettes. Command + Enter and Command + Comma in OS X, or Control + Enter and Control + Comma in Windows add columns and rows to the palettes for larger vectors and matrices. For easier readability or conceptual visualization, two-dimensional outputs of matrices are always forced by embedding the specification of the matrices in the MatrixForm commands. Matrix multiplication and matrix-vector products must always be linked by a period (.). The arguments of functions such as f[x] must be enclosed in square brackets ([]). The metavariables used to define functions must be followed by underscores (f[x_]:=). The two sides of equations written in Mathematica must be separated by double equal signs (==). Single equal signs are used for definitions, the naming of objects, and similar purposes. Mathematica commands can be written on several lines, written in the same cell and separated by pressing Enter. The command Shift + Enter evaluates a cell. Mathematica notebooks can be evaluated globally by selecting the Evaluation > Evaluate Notebook menu item. Conversely, all Mathematica outputs in a notebook can be removed at once by selecting the Cell > Delete All Output menu item. In addition to the material available in the Help file, relevant resources can also be found on the Internet by simply typing a topic of interest followed by “with mathematica.” However, some of the results found on the Internet no longer apply to recent versions of Mathematica. Only input cells can be used for computations. Depending on the document style, input cells are the default cells of Mathematica notebooks and the new cell selection automatically begins a new input cell. All cell types available in a particular notebook are listed in the Toolbar associated with a particular notebook style. Since lists of numbers correspond to row vectors and rows of matrices, the transpose is often required when working with columns. However, vectors cannot be transposed since Mathematica is designed to recognize from the context whether a row or column vector is required for a specific calculation. If column vectors are explicitly required, curly brackets must be used to separate the elements of the vectors. The examples included in this guide are designed to make it easy to understand the Mathematica syntax required for linear algebra. Mathematica and the Wolfram Research website contain tutorials that facilitate the learning of Mathematica basics. To replicate some of the material in this guide it may be necessary to click on Enable Dynamics if it appears at the top of the active Mathematica document.
Matrices
Most items in this guide involve matrices. In the Wolfram Language, a matrix is a list of lists. More specifically, a list of rows. The statement, A = {{1, 2, 3}, {4, 5, 6}}
for example, displays the “matrix” {{1,2,3},{4,5,6}} and names it A. Mathematica can be forced to display the matrix A in the customary two-dimension form in several ways. ◼ Using MatrixForm to display a matrix in two-dimensional form
MatrixForm[A = {{1, 2, 3}, {4, 5, 6}}]
123 456
However, internally, Mathematica still considers A to be a list of lists.

4 | The Linear Algebra Survival Guide

A {{1, 2, 3}, {4, 5, 6}}

◼ Using TraditionalForm to display a matrix
TraditionalForm[A = {{1, 2, 3}, {4, 5, 6}}] 123 456 ◼ Using //MatrixForm as a suffix to display a matrix in two-dimensional form
A = {{1, 2, 3}, {4, 5, 6}} /∕/∕ MatrixForm 123 456

A
123 456
By adjoining //MatrixForm to the definition of the matrix A, we force Mathematica to produce a two-dimensional output. The price we pay is that the output is no longer a computable object. Head[A]
MatrixForm

The Head command shows that instead of being a list (of lists) the output is no more than a MatrixForm.

◼ Forcing two-dimensional output in a notebook

If we evaluate the command

$Post := If[MatrixQ[#], MatrixForm[#], #] &

(1)

at the beginning of a notebook, all matrices will be displayed in two-dimensional form. However, this does not change their internal definition as lists of lists.

$Post := If[MatrixQ[#], MatrixForm[#], #] &

A = {{1, 2, 3}, {4, 5, 6}}
123 456

A
123 456

The Linear Algebra Survival Guide | 5
A+A
24 6 8 10 12
The Quit command erases the Post command and all other ad hoc definitions, assignments, and computed values. The command Clear[A] can also be used to remove an assignment without clearing the Post command. Quit[]
A = {{1, 2, 3}, {4, 5, 6}}
{{1, 2, 3}, {4, 5, 6}}
◼ Using the Mathematica Preferences to force TraditionalOutput in all notebooks.
The TraditionalOutput format of matrices and all other objects for which several output options exist can be reset globally in the Mathematica Preferences. The choice of Preferences > Evaluation > Format type of new output cells > TraditionalOutput
forces all outputs to be in TraditionalForm.
In OS X, the Preference menu is found in the Mathematica drop-down menu, whereas in Windows, it is found in the Edit drop-down menu. For matrices this means that they are output in two-dimensional form. The Quit command will not change this option back to StandardForm. ◼ Capital letters and the names of matrices
In standard linear algebra books, matrices are named with capital letters. However, Mathematica uses capital letters for built-in symbols, commands, and functions. Hence naming matrices with capital letters may be unwise. In particular, the letters C, D, E, I, K, N, and Q are reserved letters and should not be used. For easier readability and since this guide is exclusively about linear algebra, this recommendation is not followed. In most illustrations, matrices are named with the capital letter A. ◼ Matrices as two-dimensional inputs
The built-in palettes can be used to construct matrices as two-dimensional inputs:
  Palettes > Basic Math Assistant > Typesetting >
  ◼ Special symbols
Special symbols, such as the imaginary number ⅈ, can be entered in several ways. The quickest way is to type Esc ii Esc (the Escape key followed by two i’s followed by the Escape key). Similar instructions on inputting special symbols occur in various places in the chapters below. The palettes in the Palettes menu can also be used to input special symbols. Palettes > Basic Math Assistant > Typesetting > ⅈ
also inputs the symbol ⅈ. The required equivalent keyboard entry using Esc can be seen by letting the cursor rest on the symbol ⅈ in the menu. The same holds for all other displayed symbols.
Random matrices
Most illustrations and manipulations in this guide use relatively small random matrices with integer elements if the numerical content of the examples is irrelevant. This makes the examples more concrete and simplifies the notation. Two-dimensional notation for vectors and matrices is often used to conform to the standard ways of representation, although internally Mathematica uses the one-dimensional form.

6 | The Linear Algebra Survival Guide
notation for vectors and matrices is often used to conform to the standard ways of representation, although internally Mathematica uses the one-dimensional form. ◼ A two-dimensional random matrix with integer elements MatrixForm[A = RandomInteger[{0, 9}, {3, 5}]]
87304 38279 34061 produces a random 3-by-5 matrix with integer elements between 0 and 9 in two-dimensional form. However, the name A stands for the same matrix as a one-dimensional list of rows. The conversion of the input cell containing the command MatrixForm[A = RandomInteger[{0, 9}, {3, 5}]]
and the output cell containing the result of the computation
87304 38279 34061 to DisplayFormula cells ensures that the generated matrix is not changed by a second evaluation command (Shift + Enter). This change of cell types preserves the internal meaning A as a computable object. ◼ A one-dimensional output of the matrix A
A {{8, 7, 3, 0, 4}, {3, 8, 2, 7, 9}, {3, 4, 0, 6, 1}}
◼ A two-dimensional random real matrix with non-integer elements
MatrixForm[A = RandomReal[{0, 9}, {2, 3}]]
2.3487 6.53058 2.2666 6.60772 0.398364 5.2241 The resulting matrix output can be preserved by changing the Output cell in which the matrix is displayed to a DisplayFormula cell. MatrixForm[A]
2.3487 6.53058 2.2666 6.60772 0.398364 5.2241
The usual Copy and Paste commands display the matrix in the 19-place default format in which Mathematica works with the generated “real” numbers.
2.348698292382359` 6.53057615064426` 2.266599355944713` 6.607721548969266` 0.3983641332231951` 5.224099904694331`
Illustrations
Topics in illustrations are identified by a descriptive title and introduced with a square bullet (■). In most cases, the illustrations are based on random matrices with integer coefficients if this entails no loss of generality. To preserve the specific matrices generated in this way, the matrices are named and displayed in DisplayFormula cells.

The Linear Algebra Survival Guide | 7
The matrices used in the illustrations are usually embedded in a MatrixForm command. This ensures a two-dimensional output for easy viewability. But it also preserves the one-dimensional nature of Mathematica output required internally for specific calculations. The command MatrixForm[A = {{1, 2, 3}, {4, 5, 6}}]
123 456 produces a computable object named A: A {{1, 2, 3}, {4, 5, 6}}
The Head function tells us that A is a list (hence a computable object): Head[A] List
The command B = {{1, 2, 3}, {4, 5, 6}} /∕/∕ MatrixForm
123 456 produces a two-dimension object named B: B 123 456 However, the Head function confirms that B is a non-computable object called a MatrixForm. Head[B] MatrixForm
In all but a few illustrations, we avoid the suffix //MatrixForm. An illustration can be as simple as ◼ A 3-by-3 real matrix with three distinct real eigenvalues A = {{2, 3, 1}, {0, 3, 2}, {0, 0, 4}};

8 | The Linear Algebra Survival Guide

Eigenvalues[A] {4, 3, 2}
or as complicated as ◼ An orthogonal projection in ℝ2 projection = Graphics[{Arrow[{{1, 2}, {10, 2}}], Arrow[{{1, 2}, {5, 8}}],
Arrow[{{5, 8}, {5, 2}}], Arrow[{{1, 1.8}, {5, 1.8}}]}, Axes → True] 8 7 6 5 4 3

4

6

8

10

Manipulations
Many Mathematica functions and commands can be embedded in Manipulation environments controlled by parameters for varying numerical inputs. By changing the values of the parameters, the effect of numerical changes on the properties of the illustrated mathematical objects can be studied. Here is a simple example to illustrate how Manipulation works. How do changes in the elements of the following matrix A affect the eigenvalues of the matrix? MatrixForm[A = {{0, 1}, {-−1, 0}}]
01 -−1 0
Eigenvalues[A]
{ⅈ, -−ⅈ}
Let us add parameters to some of the elements of A and explore the invertibility of the resulting matrices. For what integer values of -6 ≤ a ≤ 6 and -6 ≤ b ≤ 6 does the matrix B = {{a, 1}, {-1, b}} have real eigenvalues? Manipulate[{B = {{a, 1}, {-−1, b}}, Eigenvalues[B]}, {a, -−6, 6, 1}, {b, -−6, 6, 1}]

The Linear Algebra Survival Guide | 9

a b

1

1

{{1, 1}, {-−1, -−2}},  -−1 -− 5 , -−1 + 5 

2

2

We can combine Manipulate and Eigenvalues to explore the eigenvalues of 2-by-2 matrices with integer elements. If we let a = 1 and b = - 2, for example, the manipulation produces a matrix that has real eigenvalues. By letting the Manipulate parameters range over real numbers, we can force Mathematica to produce decimal outputs.
Manipulate[{B = {{a, 1}, {-−1, b}}, Eigenvalues[B]}, {a, -−6, 6}, {b, -−6, 6}]

a b
{{{1., 1}, {-−1, -−2.}}, {-−1.61803, 0.618034}}

The N function confirms that the associated results are equal:

1

1

N -−1 -− 5 , -−1 +

2

2

{-−1.61803, 0.618034}

5 

Notation
Unless Mathematica terminology requires something else, the following notations are used in the Text cells of this guide. Vectors are named in regular bold format. The symbol v denotes a vector. Matrices are usually named in capital italic format. The symbol A denotes a matrix. A matrix with n rows and m columns is called an n-by-m matrix. Built-in Mathematica functions and commands are written in regular bold format. The random integer function and the command for solving linear systems, for example, are referred by writing RandomInteger and LinearSolve. Variables are written in italic format. The letters x, y, z, for example, denote variables. Constants are also written in italic format. The letters a, b, c, for example, denote constants. Defined functions and commands are numbered within the alphabetical items in which they occur. In Text cells, numerals are written in regular text font. The symbols 1, 2, 3, for example, denote the numbers 1, 2, and 3.
Duplications
In order to make the individual entries of this guide as readable as possible, minor duplications of illustrations became inevitable. In most cases, they are differentiated as much as possible in the presentations by emphasizing different nuances of overlapping ideas and techniques.

10 | The Linear Algebra Survival Guide
Companion Site
Interactive Mathematica Manipulation excerpts from the book, the ClassroomUtilities package, and other material complementary to this guide are published and periodically updated on the Elsevier Companion Site.

The Linear Algebra Survival Guide | 11

A

Addition of matrices
Matrices are added element by element. It works provided the matrices to be added have the same dimensions (the same number of rows and columns). In Mathematica, a plus sign between two matrices defines addition.

Properties of matrix addition

(A + B) + C = A + (B + C)

(1)

A+B=B+A

(2)

A+O=A

(3)

A + (-−1) A = O

(4)

Mathematical systems satisfying these four conditions are known as Abelian groups. For any natural number n > 0, the set of n-by-n matrices with real elements forms an Abelian group with respect to matrix addition.
The sum of an n-by-m matrix A and an n-by-m matrix B is the matrix (A + B) whose ijth element is (A[[i,j]]+ B[[i,j]]).

Illustration
◼ Addition of two matrices
MatrixForm[A = {{1, 2, 3}, {4, 5, 6}}] 123 456
Dimensions[A] {2, 3}

MatrixForm[B = {{a, b, c}, {d, e, f}}] abc def
Dimensions[B] {2, 3}

MatrixForm[A + B]
1+a 2+b 3+c 4+d 5+e 6+f

12 | The Linear Algebra Survival Guide
Dimensions[A + B] {2, 3}
◼ Sum of two 3-by-2 matrices
MatrixForm[A = {{a, b}, {c, d}, {e, f}}] ab cd ef
MatrixForm[B = {{1, 2}, {3, 4}, {5, 6}}] 12 34 56
MatrixForm[A + B] 1+a 2+b 3+c 4+d 5+e 6+f ◼ Two matrices whose sum is not defined
MatrixForm[A = RandomInteger[{0, 9}, {3, 4}]]
4987 A= 2 6 0 4 ;
0635 MatrixForm[B = RandomInteger[{0, 9}, {3, 3}]]
058 B= 9 9 6 ;
601 A+B
Thread::tdlen : Objects of unequal length in {0, 5, 8} + {4, 9, 8, 7} cannot be combined. $ Thread::tdlen : Objects of unequal length in {9, 9, 6} + {2, 6, 0, 4} cannot be combined. $ Thread::tdlen : Objects of unequal length in {6, 0, 1} + {0, 6, 3, 5} cannot be combined. $ General::stop : Further output of Thread::tdlen will be suppressed during this calculation. $ {{0, 5, 8} + {4, 9, 8, 7}, {9, 9, 6} + {2, 6, 0, 4}, {6, 0, 1} + {0, 6, 3, 5}}
◼ Sum of a nonzero and zero matrix

MatrixForm[A = {{1, 2, 3}, {4, 5, 6}}] 123 456
MatrixForm[Z = {{0, 0, 0}, {0, 0, 0}}] 000 000
A + Z ⩵ A True
◼ Sum of a matrix A and the matrix (-1) A
MatrixForm[A = {{1, 2, 3}, {4, 5, 6}}] MatrixForm[(-−1) A]
123 456 -−1 -−2 -−3 -−4 -−5 -−6
MatrixForm[A + (-−1) A] 000 000 ◼ Subtraction of two matrices
MatrixForm[A = {{1, 2, 3}, {4, 5, 6}}] 123 456
MatrixForm[B = {{a, b, c}, {d, e, f}}] abc def
MatrixForm[A + (-−1) B] 1 -− a 2 -− b 3 -− c 4 -− d 5 -− e 6 -− f
A + (-−1) B ⩵ A -− B True

The Linear Algebra Survival Guide | 13

14 | The Linear Algebra Survival Guide
Manipulation
◼ Addition of two 3-by-4 matrices Clear[a, b] A = {{1, 3 a, 0, 5}, {8, 6, 8, 3}, {3, 3, 6, 2}}; B = {{6, 2, 9, 3}, {2 b, 4, 7, 3}, {1, 5, 7, 3}}; Manipulate[Evaluate[A + B], {a, -−3, 5, 1}, {b, -−2, 4, 1}]
a b
{{7, -−7, 9, 8}, {4, 10, 15, 6}, {4, 8, 13, 5}}
We use Manipulate and Evaluate to explore the sum of two matrices. The displayed matrix is obtained by letting a = -3 and b = -2.
Adjacency matrix
The adjacency matrix of a simple labeled graph is the matrix A with A[[i,j]]= 1 or 0 according to whether the vertex vi is adjacent to the vertex vj or not. For simple graphs without self-loops, the adjacency matrix has 0s on the diagonal. For undirected graphs, the adjacency matrix is symmetric.
Illustration
◼ The adjacency matrix of an undirected graph

Graph[{1 4 2, 2 4 3, 3 4 1}]

The Linear Algebra Survival Guide | 15

The arrow / can be created by typing Esc ue Esc.
MatrixForm[AdjacencyMatrix[%]] 011 101 110 ◼ The adjacency matrix of a directed graph
Graph[{1 6 2, 2 6 3, 3 6 1}]

The arrow 0 can be created by typing Esc de Esc.
MatrixForm[AdjacencyMatrix[%]]
010 001 100
◼ The adjacency matrix of an undirected graph is symmetric

16 | The Linear Algebra Survival Guide
Graph[{1 4 2, 1 4 3, 2 4 3, 2 4 4, 3 4 4}]
MatrixForm[AdjacencyMatrix[%]] 0110 1011 1101 0110 ◼ The adjacency matrix of a directed graph can be nonsymmetric
Graph[{1 6 2, 2 6 1, 3 6 1, 3 6 2, 4 6 1, 4 6 2}]
MatrixForm[AdjacencyMatrix[%]] 0100 1000 1100 1100 ◼ The adjacency matrix of the graph with self-loops has 1s on the diagonal.

Graph[{1 4 2, 2 4 3, 3 4 1, 2 4 2}]

The Linear Algebra Survival Guide | 17

MatrixForm[AdjacencyMatrix[%]] 011 111 110 ◼ The adjacency matrix of a large graph
Graph[Table[i 6 Mod[i ^ 2, 10 ^ 3], {i, 0, 10 ^ 3 -− 1}]];
A = AdjacencyMatrix[%]

SparseArray

Specified elements: 1000 
Dimensions: {1000, 1000}

MatrixPlot[A]

1

200 400 600 800 1000

1

1

200

200

400

400

600

600

800

800

1000 1

1000 200 400 600 800 1000

18 | The Linear Algebra Survival Guide

Adjoint matrix

The adjoint of a square matrix A is the transpose of the cofactor matrix of A. In MathWorld, at http://mathworld.wolfram.com/Cofactor.html, the Mathematica functions for building adjoint matrices are defined as follows:

MinorMatrix[m_List ? MatrixQ] := Map[Reverse, Minors[m], {0, 1}]

(1)

CofactorMatrix[m_List ? MatrixQ] := MapIndexed[#1 (-−1) ^ (Plus @@ #2) &, MinorMatrix[m], {2}] (2)

These defined functions are not included in Mathematica’s function repertoire and need to be activated by typing Shift + Enter.

Illustration
◼ The cofactor matrix of a 3-by-3 matrix with integer entries
MatrixForm[A = {{1, 2, 3}, {4, 5, 6}, {7, 8, 10}}] 12 3 45 6 7 8 10
MinorMatrix[m_List ? MatrixQ] := Map[Reverse, Minors[m], {0, 1}]
CofactorMatrix[m_List ? MatrixQ] := MapIndexed[#1 (-−1) ^ (Plus @@ #2) &, MinorMatrix[m], {2}]
MatrixForm[MinorMatrix[A]] 2 -−2 -−3 -−4 -−11 -−6 -−3 -−6 -−3
MatrixForm[cfA = CofactorMatrix[A]] 2 2 -−3 4 -−11 6 -−3 6 -−3
Transpose[Det[A] Inverse[A]] ⩵ cfA True
The adjoint matrix of the given matrix A is the transpose of the cofactor matrix of A: ◼ The adjoint matrix of a 3-by-3 matrix computed from a cofactor matrix
MatrixForm[adjA = Transpose[cfA]] 2 4 -−3 2 -−11 6 -−3 6 -−3
The adjoint is equal to the determinant times the inverse of the given matrix:

The Linear Algebra Survival Guide | 19
Inverse[A] ⩵ (1 /∕ Det[A]) adjA True ◼ A general cofactor matrix Clear[a, A]; A = Array[a## &, {3, 3}]; A /∕/∕ MatrixForm
a1,1 a1,2 a1,3 a2,1 a2,2 a2,3 a3,1 a3,2 a3,3 CofactorMatrix[A] {{-−a2,3 a3,2 + a2,2 a3,3, a2,3 a3,1 -− a2,1 a3,3, -−a2,2 a3,1 + a2,1 a3,2}, {a1,3 a3,2 -− a1,2 a3,3, -−a1,3 a3,1 + a1,1 a3,3, a1,2 a3,1 -− a1,1 a3,2}, {-−a1,3 a2,2 + a1,2 a2,3, a1,3 a2,1 -− a1,1 a2,3, -−a1,2 a2,1 + a1,1 a2,2}}
Manipulation
◼ Exploring cofactor matrices MinorMatrix[m_List ? MatrixQ] := Map[Reverse, Minors[m], {0, 1}] CofactorMatrix[m_List ? MatrixQ] := MapIndexed[#1 (-−1) ^ (Plus @@ #2) &, MinorMatrix[m], {2}] Manipulate[MatrixForm[CofactorMatrix[{{a, 2, 3}, {4, b, 6}, {7, c, 10}}]],
{a, -−3, 3, 1}, {b, -−3, 3, 1}, {c, -−3, 3, 1}]
a b c
-−12 2 9 -−29 -−51 5 21 30 1
We combine Manipulate, MatrixForm, and CofactorMatrix to explore cofactor matrices. If we let a = b = c = -3, for example, the manipulation produces the cofactor matrix of the matrix {{-3, 2, 3}, {4, -3, 6}, {7, -3, 10}}.
If a matrix A represents a linear transformation T : V ⟶ V on an inner product space V in an orthonormal basis B, the transpose of A represents a linear transformation T *⋆ : V ⟶ V called the adjoint of T. The transformations T and T *⋆ are linked by the equation <T[u], v> = <u, T *⋆[v] > for all vectors u and v in V.

20 | The Linear Algebra Survival Guide
Adjoint transformation
If a matrix A represents a linear transformation T : V ⟶ V on an inner product space V in an orthonormal basis B, the transpose of A represents a linear transformation T *⋆ : V ⟶ V called the adjoint of T. The transformations T and T *⋆ are linked by the equation <T[u], v> = <u, T *⋆[v] > for all vectors u and v in V.
Illustration
Clear[x, y, z, u, v, w] MatrixForm[A = {{2, 1, 7}, {3, 5, 8}, {6, 4, 2}}]
217 358 642 MatrixForm[B = Transpose[A]] 236 154 782 With respect to the Euclidean inner product (the dot product), the transformation T and its adjoint T*⋆ are related by the equation Expand[Dot[A.{x, y, z}, {u, v, w}] ⩵ Dot[{x, y, z}, B.{u, v, w}]] True
Manipulation
◼ Exploring a transformation and its adjoint by varying the transformation matrix MatrixForm[A = {{a, 1, 7}, {3, 5, b}, {6, c, 2}}]
a17 35b 6c2 Manipulate[Evaluate[Dot[A.{1, 2, 3}, {4, 5, 6}] ⩵ Dot[{1, 2, 3}, Transpose[A].{4, 5, 6}]], {a, -−2, 2, 1}, {b, -−5, 5, 1}, {c, -−8, 8, 1}]
a b c
True

The Linear Algebra Survival Guide | 21

We combine Manipulate, Evaluate, Dot, and Transpose, to explore adjoint transformations. The manipulation shows, for example, that the transformation and its transpose determined by a = - 2, b = - 5, and c = -1 are adjoint.

Adjugate of a matrix

The adjugate of a matrix is the transpose of the cofactor matrix of the matrix. On the website http://mathematica.stackexchange.com, a Mathematica function for calculating adjugates is defined:

adj[m_] :=

Map[Reverse, Minors[Transpose[m], Length[m] -− 1], {0, 1}] *⋆

(1)

Table[(-−1) ^ (i + j), {i, Length[m]}, {j, Length[m]}];

This function is not included in the Mathematica repertoire and must to be activated by pressing Shift + Enter.

Illustration
◼ Adjugate of a 2-by-2 matrix
adj[m_] := Map[Reverse, Minors[Transpose[m], Length[m] -− 1], {0, 1}] *⋆ Table[(-−1) ^ (i + j), {i, Length[m]}, {j, Length[m]}]
A = {{a, b}, {c, d}};
adj[A] {{d, -−b}, {-−c, a}}
◼ Adjugate of a numerical 3-by-3 matrix
MatrixForm[A = {{6, 1, 9}, {3, 2, 0}, {5, 6, 8}}] 619 320 568
MatrixForm[adj[A]] 16 46 -−18 -−24 3 27 8 -−31 9
Inverse[A] ⩵ (1 /∕ Det[A]) adj[A] True

22 | The Linear Algebra Survival Guide

Affine transformation
An affine transformation is an invertible matrix transformation from ℝ2 to ℝ2, followed by a translation.
Illustration
If the translation vector is zero, then the affine transformation is simply an invertible matrix transformation. ◼ An affine transformation defined by an invertible matrix without a translation Clear[A, u]
MatrixForm[A = {{3, 2}, {9, 1}}] 32 91
Det[A] -−15

affineA[u_] := A.u

u = {1, 2};

affineA[u] {7, 11}

◼ An affine transformation defined by an invertible matrix together with a translation Clear[u, v, A] A = RandomInteger[{0, 9}, {2, 2}];

32

A=

;

91

affine[u_, v_] := A.u + v

u = {1, 2}; v = {3, 4};

affine[u, v] {10, 15}

Mathematica has a built-in function for affine transformations: ◼ An affine transformation defined with the AffineTransform command A = RandomInteger[{0, 9}, {2, 2}];

32

3

A=

;b=

;

91

4

The Linear Algebra Survival Guide | 23

tA = AffineTransform[{A, {3, 4}}]
32 3 TransformationFunction 9 1 4 
00 1
tA[{1, 2}] {10, 15}

32

0

B=

;b=

;

91

0

tB = AffineTransform[{B, {0, 0}}]

32 0 TransformationFunction 9 1 0 
00 1

tB[{1, 2}] {7, 11}

◼ An affine transformation from ℝ2 to ℝ2 defined in two-dimensional matrix notation

12 x

5

f[{x_, y_}] :=

34

.y

+

; 6

MatrixForm[f[{0, 0}]]

5 6

MatrixForm[f[{3, 4}]]
16 31

5

16

MatrixForm

+

⩵ f[{0, 0}] + f[{3, 4}]

6

31

True

◼ An affine transformation defined with the AffineTransform function t = AffineTransform[{{{a1,1, a1,2}, {a2,1, a2,2}}, {b1, b2}}]; t[{x, y}] {{{0}, {0}}1 + x a1,1 + y a1,2, {{0}, {0}}2 + x a2,1 + y a2,2}

A = {{1, 2}, {3, 4}}; b = {5, 6};

24 | The Linear Algebra Survival Guide
t = AffineTransform[{A, b}]; t[{7, 8}] {28, 59}
With the help of homogeneous coordinates, a matrix multiplication and the addition of a translation vector can be combined into a single operation. Here is how. Embed ℝ2 into ℝ3 and redefine matrix multiplication. ◼ Using homogeneous coordinates to combine a matrix multiplication and the addition of a translation vector Clear[A, b, x, y] A = {{1, 2}, {3, 4}}; b = {5, 6}; A.{x, y} + b {5 + x + 2 y, 6 + 3 x + 4 y}
◼ Converting the 2-by-2 matrix A and the vector b to a 3-by-3 matrix hA and a 3-by-3 matrix hb MatrixForm[hA = {{1, 2, 0}, {3, 4, 0}, {0, 0, 1}}]
120 340 001 MatrixForm[hb = {{1, 0, 5}, {0, 1, 6}, {0, 0, 1}}] 105 016 001
◼ Combining the matrix multiplication and translation into two matrix multiplications hb.hA.{x, y, 1} {5 + x + 2 y, 6 + 3 x + 4 y, 1}
◼ Projecting the resulting vector from ℝ3 to ℝ2 p[{u_, v_, w_}] := {u, v} p[{5 + x + 2 y, 6 + 3 x + 4 y, 1}] {5 + x + 2 y, 6 + 3 x + 4 y}
As we can see, the two results are identical: p[{5 + x + 2 y, 6 + 3 x + 4 y, 1}] == A.{x, y} + b True

The Linear Algebra Survival Guide | 25

◼ A general affine transformation form defined with the affine transform

T = AffineTransform[{{{a1,1, a1,2}, {a2,1, a2,2}}, {b1, b2}}]

a1,1 a1,2 {5, 6}1

TransformationFunction a2,1 a2,2 {5, 6}2 

00

1

T[{x, y}] {{5, 6}1 + x a1,1 + y a1,2, {5, 6}2 + x a2,1 + y a2,2}

T = AffineTransform[{{{1, 2}, {3, 4}}, {5, 6}}]
12 5 TransformationFunction 3 4 6 
00 1
T[{3, 2}] {12, 23}

{{1, 2}, {3, 4}}.{3, 2} + {5, 6} == T[{3, 2}] True

A clockwise rotation can be represented both ways, as a matrix multiplication and as an affine transformation. ◼ A clockwise rotation represented by a matrix multiplication

Cos[Pi /∕ 3] Sin[Pi /∕ 3] cwr =
-−Sin[Pi /∕ 3] Cos[Pi /∕ 3]

13

31

 ,

, -− , 

22

22

cwr.{1, 0}

1

3

 , -− 

2

2

◼ A clockwise rotation represented by an affine transformation

cwt = AffineTransform[{{{Cos[Pi /∕ 3], Sin[Pi /∕ 3]}, {-−Sin[Pi /∕ 3], Cos[Pi /∕ 3]}}, {0, 0}}]

TransformationFunction

1 2
3
-−
2
0

3
0
2

1



0

2

01

26 | The Linear Algebra Survival Guide

cwt[{1, 0}]

1

3

 , -− 

2

2

◼ A counterclockwise rotation represented by a matrix multiplication

Cos[Pi /∕ 3] -−Sin[Pi /∕ 3] ccwr =
Sin[Pi /∕ 3] Cos[Pi /∕ 3]

1

3

31

 , -− ,  , 

2

2

22

ccwr.{1, 0}

13

,



22

◼ A counterclockwise rotation represented by an affine transformation

ccwt = AffineTransform[{{{Cos[Pi /∕ 3], Sin[Pi /∕ 3]}, {-−Sin[Pi /∕ 3], Cos[Pi /∕ 3]}}, {0, 0}}]

TransformationFunction

1 2
3
-−
2
0

3
0
2

1

 0

2

01

ccwt[{1, 0}]

1

3

 , -− 

2

2

◼ A shear along the x-axis

A shear along the x- and y-axis can be represented both ways, as a matrix multiplication and as an affine transformation.

s = 5; v = {3, -−7};

shear1[s_, v_] := {{1, s}, {0, 1}}.v

shear1[s, v] {-−32, -−7}

shear2 = AffineTransform[{{{1, s}, {0, 1}}, {0, 0}}]
15 0 TransformationFunction 0 1 0 
00 1

The Linear Algebra Survival Guide | 27

shear2[{3, -−7}] {-−32, -−7} ◼ A shear along the y-axis s = 5; v = {3, -−7}; shear3[s_, v_] := {{1, 0}, {s, 1}}.v shear3[s, v] {3, 8}

shear4 = AffineTransform[{{{1, 0}, {s, 1}}, {0, 0}}]
10 0 TransformationFunction 5 1 0 
00 1

shear4[{3, -−7}] {3, 8}

◼ A reflection about y = x represented by a matrix and an affine transform A reflection about the lines y = x can also be represented both ways, as a matrix multiplication and an affine transformation.

01

MatrixFormreflyx =

;

10

point = {Cos[Pi /∕ 3], Sin[Pi /∕ 3]}

13

,



22

reflyx.point

31  ,
22
t = AffineTransform[{reflyx, {0, 0}}]
01 0 TransformationFunction 1 0 0 
00 1
t[point]

31  ,
22

28 | The Linear Algebra Survival Guide
Algebraic multiplicity of an eigenvalue
The number of repetitions of a linear factor (t - λ𝜆) in the factorization of the characteristic polynomial p[A, t] of a matrix A into linear factors is called the algebraic multiplicity of the eigenvalue λ𝜆.
Illustration
MatrixForm[A = UpperTriangularize[RandomInteger[{0, 3}, {5, 5}]]];
13100 03201 A= 0 0 0 1 0 ; 00031 00003 Clear[t]
p[A, t] = CharacteristicPolynomial[A, t] -−27 t + 54 t2 -− 36 t3 + 10 t4 -− t5
Factor[p[A, t]] -−(-−3 + t)3 (-−1 + t) t
Eigenvalues[A] {3, 3, 3, 1, 0}
The algebraic multiplicity of eigenvalue 1 is 1, and that of the eigenvalues 0 and 3 is 2. ◼ Algebraic multiplicities of eigenvalues
12031 00200 A= 0 0 0 2 1 ; 000aa 00003 p[A, t] = CharacteristicPolynomial[A, t] (3 -− t) a t2 -− t3 -− a t3 + t4

The Linear Algebra Survival Guide | 29

Factor[p[A, t]] -−(-−3 + t) (-−1 + t) t2 (-−a + t) Eigenvalues[A] {3, 1, 0, 0, a}
Manipulation
◼ Algebraic multiplicities of eigenvalues A = {{1, 2, 0, 3}, {0, 0, 2, 0}, {0, 0, a, 2}, {0, 0, a, a}}; Manipulate[Evaluate[Factor[CharacteristicPolynomial[A, t]]], {a, -−2, 2, 1}]

a (-−1 + t) t 8 + 4 t + t2

If we combine Manipulate, Evaluate, Factor, and CharacteristicPolynomial and let a = - 2, the manipulation displays the factored characteristic polynomial of the generated matrix and implies that the multiplicities of the eigenvalues of the matrix are 1, 1, and 2.

Angle
An angle is a measure of revolution, expressed in either degrees or radians. An angle θ𝜃 between two vectors u and v, expressed in radians, is the value of the function ArcCos[θ𝜃 ] where Cos[θ𝜃 ] is the cosine determined by u and v. 1 revolution = 360 degrees = 2 π𝜋 radians

2π

360

f[degrees_] :=

degrees; g[radians_] :=

radians;

360

2π

{f[360], g[2 π]}

{2 π, 360}

Illustration
◼ Angle between two vectors with respect to the Euclidean norm
Dot[u, v] cos[u_, v_] :=
Sqrt[Dot[u, u]] Sqrt[Dot[v, v]]

30 | The Linear Algebra Survival Guide

u = {3, 4}; v = {1, 5}; cos[u, v]
23 5 26 radians = N[ArcCos[cos[u, v]]] 0.446106

degrees = g[radians] 25.56

◼ Angle between two vectors with respect to a non-Euclidean norm A = {{5, 1}, {2, 7}};

MatrixForm[AAt = A.Transpose[A]]
26 17 17 53

{SymmetricMatrixQ[AAt], PositiveDefiniteMatrixQ[AAt]} {True, True}

The following command defines an inner product between vectors u and v: 〈u_, v_〉 := u.AAt.v

cos[u_, v_] :=

〈u, v〉 〈u, v〉 〈v, v〉

u = {3, 2}; v = {1, 9};

cos[u, v]

61 185
radians = N[ArcCos[cos[u, v]]] 0.959144

degrees = g[radians] 54.9549

The Linear Algebra Survival Guide | 31
◼ The angle between two vectors in ℝ3 u = {1, 2, 3}; v = {4, 5, 6}; cosine = (Dot[u, v]) /∕ (Norm[u] Norm[v]); angle = N[ArcCos[cosine]] 0.225726 This tells us that the angle between the vectors u and v is 0.225726 radians. In degrees, we get angledeg = g[angle] 12.9332
Area of a parallelogram
If u and v are two vectors in ℝ3, then the Euclidean norm Norm[Cross[u, v]] of the cross product of u and v is the area of the parallelogram determined by u and v.
Illustration
◼ Area of a parallelogram u = {1, 2, 3}; v = {4, 5, 6}; area = Norm[Cross[u, v]] 36
Manipulation
◼ Exploring the areas of parallelograms Manipulate[Norm[Cross[{1, a, 3}, {b, 5, 6}]], {a, -−3, 3, 1}, {b, -−2, 2, 1}]
a b
3 26

32 | The Linear Algebra Survival Guide
We combine Manipulate, Norm, and Cross to explore the area of parallelograms. If we let a = 2 and b = - 2, the manipulation produces the area of the parallelogram determined by the vectors {1, 2, 3} and {-2, 5, 6}.
Area of a triangle
The area of the triangle described by three points P (p1, p2, p3), Q(q1, q2, q3), and R(r1, r2, r3 ) in ℝ3 is half the area of the parallelogram determined by vectors uPQ = {p1 -− q1, p2 -− q2, q3 -− q3} and vPR = {p1 -− r1, p2 -− r2, p3 -− r3}
Illustration
◼ Area of a triangle described by the points P1 = {1, 2, 3}, P2 = {-−2, 1, 4}, and P3 = {5, 4, 2}
P1 = {1, 2, 3}; P2 = {-−2, 1, 4}; P3 = {5, 4, 2};
u = P1 -− P2; v = P1 -− P3;
1 area = Norm[Cross[u, v]]
2 3 2
In the plane, the formula for the area of a triangle is half the product of the base and height of the triangle. ◼ Area = 1 (Base × Height)
2
v1 = {0, 0}; v2 = {6, 0}; v3 = {3, 2};
1 area = (6 × 2)
2 6
By putting the z-coordinate equal to zero, we can use the norm and cross product functions to find the areas of triangles in the plane. ◼ Area = half the norm of the cross product of the vectors {6, 0, 0} and {3, 2, 0}
1 area = Norm[Cross[{6, 0, 0}, {3, 2, 0}]]
2 6
Manipulation
◼ Exploring the area of triangles

The Linear Algebra Survival Guide | 33
1 Manipulate Norm[Cross[{6, a, 0}, {3, 2 b, 0}]], {a, -−2, 2, 1}, {b, -−2, 2, 1}
2
a b
15
We combine Manipulate, Norm, and Cross to explore the areas of triangles. If we let a = - 2 and b = 2, for example, the manipulation shows that the area of the triangle determined by the vectors {6, -2, 0} and {3, 4, 0} is 15.
Array
Mathematica handles huge arrays of numeric, symbolic, textual, or any other data, with any dimension or structure. Vectors (lists of data) and matrices (two-dimensional lists of data) are special cases of arrays.
Illustration
◼ A one-dimensional array (vector in ℝ5) generated by the squaring function v = Array#2 &, 5 {1, 4, 9, 16, 25} ◼ A two-dimensional array (a 3-by-4 matrix) generated by the square-root function MatrixForm[A = Array[Sqrt[#] &, {3, 4}]]
1111 2222 3333 ◼ A 2-by-2 array with polynomial elements Clear[t] A11 = RandomInteger[{0, 9}, {2, 2}] A11 = {{2, 2}, {6, 5}}; cp11 = CharacteristicPolynomial[A11, t] -−2 -− 7 t + t2

34 | The Linear Algebra Survival Guide

A12 = RandomInteger[{0, 9}, {2, 2}] A12 = {{1, 5}, {9, 5}}; cp12 = CharacteristicPolynomial[A12, t] -−40 -− 6 t + t2 A21 = RandomInteger[{0, 9}, {2, 2}] A21 = {{2, 6}, {0, 4}}; cp21 = CharacteristicPolynomial[A21, t] 8 -− 6 t + t2 A22 = RandomInteger[{0, 9}, {2, 2}] A22 = {{2, 9}, {0, 0}}; cp22 = CharacteristicPolynomial[A22, t]; MatrixForm[A = {{cp11, cp12}, {cp21, cp22}}]
-−2 -− 7 t + t2 -−40 -− 6 t + t2 8 -− 6 t + t2 -−2 t + t2

Arrow
Vectors are frequently represented graphically by arrows in a Cartesian coordinate system to illustrate their length and direction. The Mathematica Arrow function can be used to represent arrows in two- and three-dimensional coordinate systems.

Illustration

◼ An arrow in ℝ2

Graphics[Arrow[{{0, 0}, {1, .2}}], Axes → True]

0.20 0.15 0.10 0.05

0.2

0.4

0.6

0.8

1.0

◼ An arrow in ℝ3

The Linear Algebra Survival Guide | 35
Graphics3D[Arrow[{{.1, .2, .25}, {.2, .75, 0}}], Axes → True]
0.6 0.4
0.2 0.2 0.1
0.0 0.10
0.15 0.20
Manipulation
◼ Arrows in ℝ2 Manipulate[Graphics[Arrow[{{a, b}, {1, .2}}], Axes → True], {a, -−2, 2}, {b, -−3, 3}]

36 | The Linear Algebra Survival Guide

a b

-−2.0

-−1.5

-−1.0

-−0.5

-−0.5

0.5

1.0

-−1.0

-−1.5

-−2.0

-−2.5

-−3.0
We combine Manipulate, Graphics, and Arrow to explore the representation of vectors as arrows. If we let a = - 2 and b = - 3, for example, the manipulation displays an arrow in ℝ2 starting at {-2, -3} and ending at {1, .2}.
Augmented matrix
The augmented matrix of a linear system is the matrix of the coefficients of the variables of the system and the vector of constants of the system.
Illustration
◼ The augmented matrix of a linear system in three variables and two equations

The Linear Algebra Survival Guide | 37
system = {3 x + 5 y -− z ⩵ 1, x -− 2 y + 4 z ⩵ 5}; MatrixForm[augA = {{3, 5, -−1, 1}, {1, -−2, 4, 5}}]
3 5 -−1 1 1 -−2 4 5 ◼ Combining the coefficient matrix and a constant vector using ArrayFlatten MatrixForm[A = {{3, 5, -−1}, {1, -−2, 4}}] 3 5 -−1 1 -−2 4 MatrixForm[v = {{1}, {5}}] 1 5 MatrixForm[augA = ArrayFlatten[{{A, v}}]] 3 5 -−1 1 1 -−2 4 5 ◼ Combining the coefficient matrix and a constant vector using CoefficientArrays and ArrayFlatten system = {3 x + 5 y -− z ⩵ 1, x -− 2 y + 4 z ⩵ 5}; A = Normal[CoefficientArrays[system, {x, y, z}]][[2]]; v = {{1}, {5}}; MatrixForm[augA = ArrayFlatten[{{A, v}}]] 3 5 -−1 1 1 -−2 4 5 ◼ Combining the coefficient matrix and a constant vector using Join system = {3 x + 5 y -− z ⩵ 1, x -− 2 y + 4 z ⩵ 5}; A = {{3, 5, -−1}, {1, -−2, 4}}; v = {{1}, {5}}; MatrixForm[Join[A, v, 2]] 3 5 -−1 1 1 -−2 4 5 The ClassroomUtilities add-on package for Mathematica contains two elegant converters for alternating between linear systems and augmented matrices.

38 | The Linear Algebra Survival Guide

◼ Using the ClassroomUtilities Needs["ClassroomUtilities`"] eqns = {3 x + y ⩵ 4, x -− y ⩵ 1}; vars = {x, y}; MatrixForm[CreateAugmentedMatrix[eqns, vars]]
3 1 -−4 1 -−1 -−1 Clear[x, y] A = {{3, 1, -−4}, {1, -−1, -−1}}; vars = {x, y}; CreateEquations[A, vars] {3 x + y ⩵ -−4, x -− y ⩵ -−1}
Manipulation
◼ Linear systems and their augmented matrices Manipulate[{{3 x + 5 y -− z c ⩵ a, x -− 2 y + 4 z ⩵ b},
MatrixForm[{{3, 5, -−c, a}, {1, -−2, 4, b}}]}, {a, 0, 5, 1}, {b, 0, 5, 1}, {c, -−5, 5, 1}]

a b c

3 5 53

{3 x + 5 y + 5 z ⩵ 3, x -− 2 y + 4 z ⩵ 2},



1 -−2 4 2

We use Manipulate and MatrixForm to explore the connection between linear systems and their augmented matrices. If we let a = 3, b = 2, and c = - 5, for example, the manipulation displays the resulting linear system and its augmented matrix.

The Linear Algebra Survival Guide | 39
B
Back substitution
A linear system Av = b can sometimes be solved by decomposing the coefficient matrix A into a product LU, where L is a lower-triangular and U is an upper-triangular matrix. The system can then be solved by solving the systems Lw = b and Uv = w. Since U is upper-triangular, the system Uv = w can be solved by back substitution. (The associated system Lw = b is solved by forward substitution.)
Illustration
◼ Solving a linear system in three equations and three variables system = {eq1, eq2, eq3} = {x -− 2 y + 5 z ⩵ 12, 4 y + 3 z ⩵ 5, 2 z ⩵ 4}; solutionz = Reduce[eq3] z ⩵ 2
solutiony = Reduce[eq2 /∕. {z → 2}] 1
y ⩵ -− 4
solutionx = Reduce[eq1 /∕. {z → 2, y → -−1 /∕ 4}] 3
x ⩵ 2
Flatten[Reduce[system, {x, y, z}]] ⩵ (solutionx && solutiony && solutionz) True
Band matrix
A band matrix is a sparse matrix whose nonzero elements occur only on the main diagonal and on zero or more diagonals on either side of the main diagonal.
Illustration
◼ A diagonal matrix is a band matrix MatrixForm[A = DiagonalMatrix[{1, 2, 3, 0, 5, 5}]];

40 | The Linear Algebra Survival Guide

100000

020000

003000

A=

;

000000

000050

000005

◼ A Hessenberg matrix is a band matrix

A = {{1, 2, 3, 4}, {5, 6, 7, 8}, {9, 10, 11, 12}, {13., 14, 15, 16}};

{p, h} = HessenbergDecomposition[A];

MatrixForm[Chop[h]]

1.

-−5.3669 0.443129 0

-−16.5831 33.0873 -−9.55746 0

0

-−2.20899 -−0.0872727 0

0

0

0

0

◼ A shear matrix is a band matrix

A = {{1, a}, {b, 1}};

A.{x, y} {x + a y, b x + y}

◼ Two-dimensional display of a band matrix diagonal = {0, 2, 1, 5, 8, 0, 8, 9};

A = SparseArray[DiagonalMatrix[diagonal]]

SparseArray

Specified elements: 6 
Dimensions: {8, 8}

A /∕/∕ MatrixForm
00000000 02000000 00100000 00050000 00008000 00000000 00000080 00000009
◼ A Jordan block is a band matrix

The Linear Algebra Survival Guide | 41
A = {{27, 48, 81}, {-−6, 0, 0}, {1, 0, 3}}; {s, j} = JordanDecomposition[A]; MatrixForm[j]
60 0 0 12 1 0 0 12
Basic variable of a linear system
The variables of a linear system Av = b corresponding to the pivot columns of the coefficient matrix A are the basic variables of the system.
Illustration
◼ Basic variables of a linear system system = {3 x + 4 y -− z ⩵ 9, x + y + 4 z ⩵ 1}; A = {{3, 4, -−1}, {1, 1, 4}}; MatrixForm[RowReduce[A]]
1 0 17 0 1 -−13 Since the first and second columns of A are the pivot columns of the given linear system, the variables x and y are basic variables of the system.
Basis of a vector space
A basis of a vector space is a linearly independent subset of vectors that spans the space.
Illustration
◼ A basis for ℝ3 basis = {{0, 3, 2}, {5, 8, 1}, {5, 0, 6}}
◼ Linear independence Solve[a {0, 3, 2} + b {5, 8, 1} + c {5, 0, 6} ⩵ {0, 0, 0}] {{a → 0, b → 0, c → 0}}
◼ Span

42 | The Linear Algebra Survival Guide

span = Solve[a {0, 3, 2} + b {5, 8, 1} + c {5, 0, 6} ⩵ {x, y, z}, {a, b, c}]

1

1

1

a →

(-−48 x + 25 y + 40 z), b →

(18 x + 10 y -− 15 z), c →

(13 x -− 10 y + 15 z)

155

155

155

span /∕. {x → 1, y → 1, z → 1}

17

13

18

a → , b → , c → 

155

155

155

17

13

18

coordinates = a → , b → , c → 

155

155

155

17

13

18

a → , b → , c → 

155

155

155

{1, 1, 1} == a {0, 3, 2} + b {5, 8, 1} + c {5, 0, 6} /∕. coordinates {True}

The columns or rows of an invertible matrix are linearly independent and every vector in the space can be written as a unique linear combination of the columns or the rows of the matrix, provided that the order of the columns or rows is respected. Therefore the columns or rows form a basis of the column space or row space of the matrix. ◼ An invertible matrix considered as a basis for ℝ3
MatrixForm[A = RandomInteger[{0, 9}, {3, 3}]];

585 A= 9 9 0 ;
806
Det[A]
-−522

Solve[{x, y, z} ⩵ a A[[1]] + b A[[2]] + c A[[3]], {x, y, z}] {{x → 5 a + 9 b + 8 c, y → 8 a + 9 b, z → 5 a + 6 c}}

{x, y, z} /∕. {{x → 5 a + 9 b + 8 c, y → 8 a + 9 b, z → 5 a + 6 c}} {{5 a + 9 b + 8 c, 8 a + 9 b, 5 a + 6 c}}
The result shows that every vector {x, y, z} in ℝ3 can be written as a linear combination of the rows of the matrix A. To show that the rows form a basis, the uniqueness of the linear combination still needs to be verified. ◼ The rank (number of linearly independent rows or columns) of a matrix
A = {{1, 2, 3}, {4, 5, 6}, {7, 8, 10}};

The Linear Algebra Survival Guide | 43
MatrixRank[A] 3
This tells us that the three rows of the matrix m are linearly independent. So are the three columns. lc = {x, y, z} == a A[[1]] + b A[[2]] + c A[[3]] {x, y, z} ⩵ {a + 4 b + 7 c, 2 a + 5 b + 8 c, 3 a + 6 b + 10 c}
Solve[lc, {x, y, z}] {{x → a + 4 b + 7 c, y → 2 a + 5 b + 8 c, z → 3 a + 6 b + 10 c}}
Since the rows of the matrix A are linearly independent, each vector {x, y, z} in the space ℝ3 can be written as a unique linear combination of the rows of A. Randomly generated n-by-n matrices with real entries are often (usually) invertible. Their rows and columns can therefore be used as bases for ℝn. ◼ A basis for ℝ5 derived from randomly generated matrices MatrixForm[A = RandomInteger[{0, 9}, {5, 5}]];
97935 39257 A= 4 7 5 6 1 ; 73441 12531 MatrixRank[A] 5
Reduce[a A[[1]] + b A[[2]] + c A[[3]] + d A[[4]] + e A[[5]] ⩵ 0, {a, b, c, d, e}] a ⩵ 0 && b ⩵ 0 && c ⩵ 0 && d ⩵ 0 && e ⩵ 0
This shows that the rows of the matrix A are linearly independent and form a basis for ℝ5. Similarly, the calculation Reduce[a A[[All,1]] + b A[[All,2]] + c A[[All,3]] + d A[[All,4]] + e A[[All,5]] ⩵ 0, {a, b, c, d, e}] a ⩵ 0 && b ⩵ 0 && c ⩵ 0 && d ⩵ 0 && e ⩵ 0
shows that the columns of A are linearly independent and also form a basis for ℝ5.
Manipulation
◼ Linearly dependent and independent vectors Manipulate[Evaluate[Reduce[a {1, 2, 3} + b {4, 5, 6 } + c {7, 8, 9 d} ⩵ {0, 0, 0}, {a, b, c}]],
{d, -−2, 2, 1}]

44 | The Linear Algebra Survival Guide
d
b ⩵ -−2 a && c ⩵ a
We combine Manipulate and Reduce to explore the linear dependence and independence of vectors in ℝ3. The displayed window shows, for example, that if we let d = 1, the generated vectors are linearly dependent. On the other hand, if we let d = 2, the generated vectors are linearly independent.
d
a ⩵ 0 && b ⩵ 0 && c ⩵ 0
◼ Exploring the invertibility of a matrix Manipulate[Inverse[{{a, b}, {2, 3}}], {a, -−2, 2, 1}, {b, -−2, 2, 1}]
a b
3 -− , -−1, {1, 1}
2
We combine Manipulate and Inverse to explore the invertibility of matrices. If we let a = b = - 2, for example, the window displays the inverse of the generated matrix.
Bijective linear transformation
A linear transformation T : V ⟶ W from a vector space V to a vector space W is bijective if it is both injective (one-to-one) and surjective (onto). Bijective transformations are invertible in the sense that there exists a linear transformation S : W ⟶ V for which S[T[v]] = v and T[S[w]] = w for all v in V and w in W.
Illustration
◼ A bijective linear transformation from ℝ2 to ℝ2 T[{x_, y_}] := {{Cos[θ], Sin[θ]}, {-−Sin[θ], Cos[θ]}}.{x, y}

The Linear Algebra Survival Guide | 45

S[{x_, y_}] := {{Cos[θ], -−Sin[θ]}, {Sin[θ], Cos[θ]}}.{x, y}

θ = π /∕ 3; x = 1; y = 1;

{T[{x, y}], S[{x, y}]}

1 31 3

1 31 3

 +

, -−

,  -−

,+



2 22 2

2 22 2

Simplify[Composition[S, T][{x, y}] ⩵ {x, y}] True

Simplify[Composition[T, S][{x, y}] ⩵ {x, y}] True

Bilinear functional
If V is a real vector space, then a function T[u, v] from V⨯V with values in the real vector space ℝ is a (real) bilinear functional if it is linear both in u and v. A similar definition leads to the idea of a complex bilinear functional. In the context of linear functionals, the values T[u, v] are usually written as 〈u, v〉. Bilinearity is defined by the following properties:

Properties of bilinear functionals

〈a u + b v, w〉 = a 〈u, w〉 + b 〈u, w〉

(1)

〈u, c v + d w〉 = c 〈u, v〉 + d 〈u, w〉

(2)

Illustration
◼ The trace as a bilinear functional on 2-by-2 real matrices Clear[a, b, c, d, e, f, g, h] A = {{a, b}, {c, d}}; B = {{e, f}, {g, h}}; {Tr[A], Tr[B]} {a + d, e + h}

Tr[3 A + 5 B] ⩵ Expand[3 Tr[A] + 5 Tr[B]] True ◼ Multiplication as a bilinear functional on ℝ2 Clear[x, y, z, a, b, c, d]

46 | The Linear Algebra Survival Guide
T[x_, y_] := x y left = Expand[T[a x + b y, z]] ⩵ a T[x, z] + b T[y, z]; right = Expand[T[x, c y + d z]] ⩵ c T[x, y] + d T[x, z]; a = b = 1; x = 5; y = -−6; z = 3; {T[a x + b y, z], T[x, z] + b T[y, z]} {-−3, -−3} c = d = 1; x = 2; y = 7; z = -−9; {T[x, c y + d z], c T[x, y] + d T[x, z]} {-−4, -−4} ◼ The dot product as a bilinear functional on ℝ4 Clear[x, y, z, a, b, c, d, e, f, g, h, i, j, k, l] cr = CharacterRange["a", "l"] {a, b, c, d, e, f, g, h, i, j, k, l} x = {a, b, c, d}; y = {e, f, g, h}; z = {i, j, k, l}; Expand[Dot[5 x + 7 y, z] ⩵ 5 Dot[x, z] + 7 Dot[y, z]] True

The Linear Algebra Survival Guide | 47

C

Cartesian product of vector spaces

The Cartesian product U × V of two finite-dimensional vector spaces U and V consists of all pairs {u, v} of vectors u in U and v in V. The vector space operations are defined componentwise:

{u1, v1} + {u2, v2} = {u1 + u2, v1 + v2}

(1)

a {u, v} = {a u, a v}

(2)

Illustration
◼ The Cartesian product of two vector spaces basisU = {{1, 0}, {0, 1}} {{1, 0}, {0, 1}}

basisV = {{1, 2, 3}, {0, 4, 1}} {{1, 2, 3}, {0, 4, 1}}

basisUV = Tuples[{basisU, basisV}] {{{1, 0}, {1, 2, 3}}, {{1, 0}, {0, 4, 1}}, {{0, 1}, {1, 2, 3}}, {{0, 1}, {0, 4, 1}}}

Cauchy–Schwarz inequality

If u and v are two vectors in an inner product space V, then the Cauchy–Schwarz inequality states that for all vectors u and v in V,

=〈u, v〉>2 ≤ Dot[〈u, u〉, 〈v, v〉]

(1)

The bilinear functional <u, v> is the inner product of the space V. The inequality becomes an equality if and only if u and v are linearly dependent.

Illustration
◼ The Cauchy–Schwarz inequality of two vectors in ℝ3 u = {1, 2, 3}; v = {-−4, 5, 6};
Abs[Dot[u, v]]2 ≤ Dot[u, u] Dot[v, v] True

48 | The Linear Algebra Survival Guide
Abs[Dot[u, v]]2, Dot[u, u] Dot[v, v] {576, 1078} If w is a multiple of u, for example, then the inequality becomes an equality: w = 3 u; Abs[Dot[u, w]]2 == Dot[u, u] Dot[w, w] True ◼ The Cauchy–Schwarz inequality of two vectors in ℂ2 u = {3 + I, -−5}; v = {5 -− 2 I, 6 + I}; Abs[Dot[u, Conjugate[v]]]2 ≤ Dot[u, Conjugate[u]] Dot[v, Conjugate[v]] True
Manipulation
◼ Cauchy–Schwarz inequalities u = {b, 2, 3}; v = {a, 5, 6}; ManipulateEvaluateAbs[Dot[u, v]]2 ≤ Dot[u, u] Dot[v, v], {a, -−2, 2}, {b, -−5, 5}
a b
True
We can combine Manipulate, Abs, and Dot to explore the Cauchy–Schwarz inequality. If we assign the values a = - 2 and b = - 5, for example, the manipulation shows that the Cauchy–Schwarz inequality holds for the vectors {- 5, 2, 3} and {- 2, 5, 6}.
Cayley–Hamilton theorem
One of the best-known properties of characteristic polynomials is that all square real or complex matrices satisfy their characteristic polynomials. This result is known as the Cayley–Hamilton theorem.

The Linear Algebra Survival Guide | 49
Illustration
◼ The Cayley–Hamilton theorem illustrated with a 3-by-3 matrix 211
A= 4 1 7 ; 530
characteristicPolynomialA = Det[A -− t IdentityMatrix[3]] 28 t + 3 t2 -− t3 charPolyA = -−MatrixPower[A, 3] + 3 MatrixPower[A, 2] + 28 A {{0, 0, 0}, {0, 0, 0}, {0, 0, 0}} This shows that the matrix A satisfies of its own characteristic polynomial.
Manipulation
◼ The Cayley–Hamilton illustrated with Manipulate 211
A= a 1 7 ; 530
cpA = CharacteristicPolynomial[A, t]
-−12 + 3 a + 24 t + a t + 3 t2 -− t3 Manipulate[
Evaluate[-−MatrixPower[A, 3] + 3 MatrixPower[A, 2] + (24 + a) A -− (12 -− 3 a) IdentityMatrix[3]], {a, -−5, 5, 1}]
a
{{0, 0, 0}, {0, 0, 0}, {0, 0, 0}}
We can use Manipulate, MatrixPower, and IdentityMatrix to show that the Cayley–Hamilton theorem holds for the generated matrices. If we let a = - 5, for example, the manipulation shows that the Cayley–Hamilton theorem holds for the generated matrix.

50 | The Linear Algebra Survival Guide

Change-of-basis matrix
See Coordinate conversion matrix

Characteristic polynomial

The characteristic polynomial p[t] of an n-by-n real or complex matrix A in the variable t is the polynomial

p[t] = Det[A -− t IdentityMatrix[n]]

(1)

The roots of p[t] are the eigenvalues of the matrix A.

Illustration
◼ The characteristic polynomial of a 5-by-5 matrix
Clear[A, t] A = RandomInteger[{0, 9}, {5, 5}];
43819 17498 A= 5 3 8 5 1 ; 23605 49868 CharacteristicPolynomial[A, t] 3206 -− 851 t -− 675 t2 -− 8 t3 + 27 t4 -− t5
◼ Using determinants to find the characteristic polynomial of a 5-by-5 matrix
Det[A -− t IdentityMatrix[5]] 3206 -− 851 t -− 675 t2 -− 8 t3 + 27 t4 -− t5
The two calculations show that
CharacteristicPolynomial[A, t] == Det[A -− t IdentityMatrix[5]] True

Cholesky decomposition
The Cholesky decomposition of a square matrix is a decomposition of a Hermitian, positive-definite matrix into a product of a lower-triangular matrix and its conjugate transpose.
Illustration
◼ Cholesky decomposition of a 2-by-2 symmetric positive-definite matrix

The Linear Algebra Survival Guide | 51

MatrixForm[A = {{2, 1}, {1, 2}}] 21 12
{HermitianMatrixQ[A], PositiveDefiniteMatrixQ[A]} {True, True}

cdA = CholeskyDecomposition[A]

1

 2 ,

, 0,

2

3 
2

ConjugateTranspose[cdA].cdA {{2, 1}, {1, 2}}

◼ Cholesky decomposition of a 3-by-3 symmetric positive-definite matrix

2 -−1 0 A = -−1 2 1 ;
0 12
{PositiveDefiniteMatrixQ[A], HermitianMatrixQ[A]}
{True, True}

MatrixForm[cdA = CholeskyDecomposition[A]]

1

2 -−

0

2

3

2

0

2

3

00

2

3

MatrixForm[ConjugateTranspose[cdA].cdA]
2 -−1 0 -−1 2 1 0 12

52 | The Linear Algebra Survival Guide
Codimension of a vector subspace
If W is a subspace of a finite-dimensional vector space V, then the codimension of the subspace W in the space V is the difference dim[V] - dim[W] between the dimension of the space V and the dimension of the subspace W.
Illustration
◼ A subspace of ℝ3 with codimension 2 in ℝ3 V = ℝ3; W = {{a, 0, b} : a, b ∈ Reals}; The dimension of V is 3 and the dimension of the subspace W is 2. Hence the codimension of W is 1. ◼ A subspace of ℝ5 with codimension 3 in ℝ5 V = ℝ5; W = span[{{1, 0, 1, 0, 1}, {0, 1, 1, 0, 0}}] W = {a {1, 0, 1, 0, 1} + b {0, 1, 1, 0, 0} : a, b ∈ ℝ} = {a, b, b, 0, a} : a, b ∈ ℝ} Since W is a subspace of ℝ5 of dimension 2, its codimension in ℝ5 is 3.
Codomain of a linear transformation
The codomain of a linear transformation T is the vector space in which the transformation takes its values. The notation T : A ⟶ B identifies the vector space A as the domain of T and the vector space B as its codomain. The range of T is the subspace of the codomain of T consisting of all values of T. If the range of T coincides with the codomain of T, the transformation T is said to be onto or surjective. If there exists a transformation S : B ⟶ A for which the composition (S · T) of S and T is the identity transformation idA : A ⟶ A on A and the composition (T · S) is the identity transformation idB : B ⟶ B, then the transformation T is said to be one-to-one or injective. If T is both onto and one-to-one, it is said to be bijective.
Illustration
◼ An injective linear transformation T : ℝ2 ⟶ ℝ3 T[{x_, y_}] := {x + y, 2 x -− y, y};
T[{1, 2}] {3, 0, 2}
The codomain of the transformation T is ℝ3. However, not every vector in ℝ3 is a value of T. For example, the vector {5, 5, 5} in ℝ3 is not a value of T. Clear[x, y, a, b, c, d] Solve[{x + y, 2 x -− y, y} ⩵ {5, 5, 5}, {x, y}] {}
◼ A surjective linear transformation T : ℝ3 ⟶ ℝ2 T[{x_, y_, z_}] := {y, x};

The Linear Algebra Survival Guide | 53

T[{-−2, 3, 5}] {3, -−2} The codomain of the transformation T is ℝ2. And, any vector {x, y} in ℝ2 is a value of T. Solve[{x, y} ⩵ T[{a, b, c}], {a, b, c}] Solve::svars : Equations may not give solutions for all "solve" variables. $ {{a → y, b → x}}
T[{a, b, c}] /∕. {a → y, b → x} {x, y} ◼ A bijective linear transformation T : ℝ[t, 3] ⟶ ℝ4 Ta_ + b_ t_ + c_ t_2 + d_ t_3 := {a, b, c, d} S[{a_, b_, c_, d_}] := a + b t + c t2 + d t3 Composition[S, T]a + b t + c t2 + d t3 a + b t + c t2 + d t3 Composition[T, S][{a, b, c, d}] {a, b, c, d}

Cofactor matrix
The cofactor matrix of a square matrix A is the matrix of cofactors of A. The cofactors cfAij are (-−1)i+ j times the determinants of the submatrices Aij obtained from A by deleting the ith rows and jth columns of A. The cofactor matrix is also referred to as the minor matrix. It can be used to find the inverse of A.

Illustration
◼ Cofactor matrix of a 3-by-3 matrix

452 A= 1 4 6 ;
706

cfA11 = (-−1)1+1 Det 4 6 ; cfA12 = (-−1)1+2 Det 1 6 ; cfA13 = (-−1)1+3 Det 1 4 ;

06

76

70

cfA21 = (-−1)2+1 Det 5 2 ; cfA22 = (-−1)2+2 Det 4 2 ; cfA23 = (-−1)2+3 Det 4 5 ;

06

76

70

54 | The Linear Algebra Survival Guide

cfA31 = (-−1)3+1 Det 5 2 ; cfA32 = (-−1)3+2 Det 4 2 ; cfA33 = (-−1)3+3 Det 4 5 ;

46

16

14

MatrixForm[cfA = {{cfA11, cfA12, cfA13}, {cfA21, cfA22, cfA23}, {cfA31, cfA32, cfA33}}]

24 36 -−28 -−30 10 35 22 -−22 11

(1 /∕ Det[A]) Transpose[cfA] ⩵ Inverse[A] True

The following Mathematica definition can be used to calculate the cofactors of a given matrix: Clear[A, i, j]

452 A= 1 4 6 ;
706
Cofactor[m_List ? MatrixQ, {i_Integer, j_Integer}] := (-−1) ^ (i + j) Det[Drop[Transpose[Drop[Transpose[m], {j}]], {i}]]
The Cofactor command is a defined command and needs to be activated before it can be used by typing Shift + Enter.
CfA = MatrixForm[Table[Cofactor[A, {i, j}], {i, 1, 3}, {j, 1, 3}]];

MatrixForm[cfA] ⩵ CfA True

The next Mathematica definitions can be used to calculate the cofactor matrix in one step: MinorMatrix[m_List ? MatrixQ] := Map[Reverse, Minors[m], {0, 1}]

CofactorMatrix[m_List ? MatrixQ] := MapIndexed[#1 (-−1) ^ (Plus @@ #2) &, MinorMatrix[m], {2}]
Both commands are defined commands and must be activated by typing Shift + Enter before they can be used. The commands are defined in MathWorld at http://mathworld.wolfram.com/Cofactor.html.
MatrixForm[MinorMatrix[A]]
24 -−36 -−28 30 10 -−35 22 22 11

Column space
The column space of a matrix A is the set of all linear combinations of the columns of A. The column space of the matrix tells us if and when a linear system Av = b has a solution v.

The Linear Algebra Survival Guide | 55

Illustration
◼ A basis for the column space of a 3-by-4 matrix
MatrixForm[A = {{1, 2, 3, 4}, {5, 6, 7, 8}, {0, 1, 3, 0}}]
1234 5678 0130

MatrixForm[B = RowReduce[A]]
1 0 0 -−5 010 9 0 0 1 -−3
The reduction shows that the first three columns of the matrix B are pivot columns. They are linearly independent and span the column space of A.
◼ Linear independence
Clear[a, b, c]

1

2

3

0

Solvea 5 + b 6 + c 7 == 0 , {a, b, c}

0

1

3

0

{{a → 0, b → 0, c → 0}}

Thus the zero vector is a linear combination of the first three columns if and only if the coefficients a, b, and c are zero. The fourth column of A is a linear combination of the first three:

1

2

3

4

Solvea 5 + b 6 + c 7 == 8 , {a, b, c}

0

1

3

0

{{a → -−5, b → 9, c → -−3}}

1

2

3

4

-−5 5 + 9 6 -− 3 7 == 8

0

1

3

0

True

◼ Spanning Clear[a, b, c, x, y, z]

56 | The Linear Algebra Survival Guide

1

2

3

x

Solvea 5 + b 6 + c 7 == y , {a, b, c}

0

1

3

z

1

1

1

a → (-−11 x + 3 y + 4 z), b → (15 x -− 3 y -− 8 z), c → (-−5 x + y + 4 z)

4

4

4

This shows that every vector {x. y, z} in the column space of the matrix A can be constructed as a linear combination of the first three columns of the matrix A.
◼ The column space of a 3-by-5 matrix determined by the pivot columns of the matrix

MatrixForm[Transpose[A = RandomInteger[{0, 9}, {5, 3}]]]

04834 A= 8 5 0 6 9 ;
27381
MatrixForm[Transpose[A]]
04834 85069 27381

MatrixForm[B = RowReduce[Transpose[A]]]

1

0

0

11
-−

272

0 1 0 43
34

0

0

1

35
-−

136

26 17 11
-−
17 14 17

This shows that the first three columns of A are the pivot columns of A. They are linearly independent and span the column space of A.
◼ Linear independence

Solve[a B[[All,1]] + b B[[All,2]] + c B[[All,3]] ⩵ {0, 0, 0}, {a, b, c}] {{a → 0, b → 0, c → 0}}

◼ The third column as a linear combination of the first three columns Clear[a, b, c]

Solve[a B[[All,1]] + b B[[All,2]] + c B[[All,3]] ⩵ {-−11 /∕ 272, 43 /∕ 34, -−35 /∕ 136}, {a, b, c}]

11

43

35

a → -− , b → , c → -− 

272

34

136

◼ The fourth column as a linear combination of the first three columns

Clear[a, b, c]

The Linear Algebra Survival Guide | 57

Solve[a B[[All,1]] + b B[[All,2]] + c B[[All,3]] ⩵ {26 /∕ 17, -−11 /∕ 17, 14 /∕ 17}, {a, b, c}]

26

11

14

a → , b → -− , c → 

17

17

17

Column vector
A column vector is a vertical list of scalars. In Mathematica, column vectors are represented as lists of singleton lists.
Illustration
◼ A column vector of height 3 in standard form Unless required to do otherwise, Mathematica outputs column vectors in standard form. columnvector = {{1}, {2}, {3}} {{1}, {2}, {3}}
◼ A column vector of height 3 in traditional form. MatrixForm[columnvector]
1 2 3

Companion matrix

The (Frobenius) companion matrix of the monic polynomial

p[t] = a0 + a1 t + ⋯ + an-−1 tn-−1 + tn

(1)

is the square matrix

0 0 … 0 -−a0

1 0 … 0 -−a1

C[p] = 0 1 … 0 -−a2

(2)

⋮ ⋮ ⋱ ⋮ ⋮

0 0  1 -−an-−1

The characteristic polynomial and the minimal polynomial of C[p] are both equal to p[t]. Sometimes the companion matrix is defined as the transpose of C[p].

Illustration
◼ Companion matrix of a monic polynomial of degree 3

58 | The Linear Algebra Survival Guide
p = t3 -− 4 t2 + 5; The following defined Mathematica function can be used to calculate the companion matrix of a polynomial: CompanionMatrix[p_, x_] := Module[{n, w = CoefficientList[p, x]}, w = -−w /∕ Last[w];
n = Length[w] -− 1; SparseArray[{{i_, n} ⧴ w[[i]], {i_, j_} /∕; i ⩵ j + 1 → 1}, {n, n}]] The command is defined in MathWorld and can be found at http://mathworld.wolfram.com/CompanionMatrix.html. MatrixForm[Normal[CompanionMatrix[p, t]]] 0 0 -−5 10 0 01 4 ◼ Companion matrix of a monic polynomial of degree 4
p = 3 + 5 t -− 7 t3 + t4; MatrixForm[A = Normal[CompanionMatrix[p, t]]]
0 0 0 -−3 1 0 0 -−5 010 0 001 7
Complex conjugate
The complex conjugate of the complex number (a + b ⅈ) is the number (a - b ⅈ).
Illustration
◼ The complex conjugates of five complex numbers numbers = {{3 -− 7 I}, {5 I}, {9}, {-−2 + 4 I}, {8 + I}} {{3 -− 7 ⅈ}, {5 ⅈ}, {9}, {-−2 + 4 ⅈ}, {8 + ⅈ}}
conjugates = Conjugate[numbers] {{3 + 7 ⅈ}, {-−5 ⅈ}, {9}, {-−2 -− 4 ⅈ}, {8 -− ⅈ}}
◼ The conjugates of a product and a quotient of complex numbers numbers = {{3 -− 7 I} {5 I}, {9} /∕ {-−2 + 4 I}}
9 9 ⅈ {35 + 15 ⅈ}, -− -− 
10 5

The Linear Algebra Survival Guide | 59
conjugates = Conjugate[numbers] 9 9 ⅈ
{35 -− 15 ⅈ}, -− +  10 5
Composition of linear transformations
If T is a linear transformation from ℝn to ℝm and S is a linear transformation from ℝm to ℝp, then the composition of T and S, denoted by (S∘T), is defined by S(T(x)) for all x in ℝn. The composite transformation (S ∘ T) is represented by the matrix product ST of the matrices S and T provided that the domain basis of S is the same as the codomain basis of T.
Illustration
◼ The composition function Composition[f, g][x, y] == f[g[x, y]] True ◼ Composition of a linear transformation from ℝ3 to ℝ2 with a linear transformation from ℝ2 to ℝ2 T[{x_, y_, z_}] := {3 x + 2 y -− 4 z, x -− 5 y + 3 x} S[{u_, v_}] := {4 u -− 2 v, 2 u + v} T[{1, 2, 3}] {-−5, -−6}
S[{-−5, -−6}] {-−8, -−16} The matrices representing the transformations T, S, and the composition of T and S are as follows: mT= Transpose[{T[{1, 0, 0}], T[{0, 1, 0}], T[{0, 0, 1}]}] {{3, 2, -−4}, {4, -5− , 0}}
mS = Transpose[{S[{1, 0}], S[{0, 1}]}] {{4, -−2}, {2, 1}}
mP = mS.mT {{4, 18, -−16}, {10, -−1, -−8}}

60 | The Linear Algebra Survival Guide

mP.{1, 2, 3} { 8, -− 16} This last operation shows that the matrix mP produces the same result as the composition of the functions T and S.

Condition number of a matrix
Various definitions of condition numbers of matrices are used to measure the impact of a relatively small change in inputs on the outputs. For example, the condition numbers associated with a linear equation Av = b measure how inaccurate the solution v will be after a small change in the coefficient matrix A. Thus the condition number of a matrix A relative to a matrix norm is defined as the norm of A divided by the norm of the inverse of A.

Illustration
◼ Condition numbers of Hilbert matrices hilbert = Table[HilbertMatrix[n], {n, 1, 5}]; conditionnumbers = Table[LinearAlgebra`MatrixConditionNumber[hilbert[[n]]], {n, 1, 5}] {1, 27, 748, 28 375, 943 656}

This example shows the rapid growth of the condition numbers of even small Hilbert matrices.
Another measure used to calculate the condition number of a matrix is the two-norm consisting of the ratio of the largest to the smallest singular value of the matrix. {u, w, v} = SingularValueDecomposition[hilbert[[5]]];

MatrixForm[N[w]]

1.56705 0.

0.

0.

0.

0. 0.208534

0.

0.

0.

0.

0. 0.0114075

0.

0.

0.

0.

0.

0.000305898

0.

0.

0.

0.

0.

3.28793 × 10-−6

In the case of the 5-by-5 Hilbert matrix, the condition number based on the two-norm is therefore

N[w][[1]][[1]] /∕ N[w][[5]][[5]] 476 607.

The Linear Algebra Survival Guide | 61

Congruence transformation
For any n-by-n symmetric matrix A and any nonsingular n-by-n matrix B, the transformation A ⟶ BTAB is a congruence transformation. Congruence transformations preserve the number of positive, negative, and zero eigenvalues.
Illustration
◼ Congruence transformation of a 5-by-5 symmetric matrix
02030 22010 A= 0 0 0 2 0 ; 31200 00000 SymmetricMatrixQ[A] True
N[Eigenvalues[A]] {4.93812, -−3.65125, 1.36381, -−0.650674, 0.}
B = {{5, 3, 3, 9, 5}, {5, 6, 4, 4, 9}, {2, 0, 3, 5, 6}, {5, 7, 0, 1, 9}, {8, 5, 6, 5, 0}};
Det[B] 3728
N[Eigenvalues[Transpose[B].A.B]]
{2299.83, -− 226.481, -− 6.48762, 5.13604, 0.} Both A and BTAB have two positive eigenvalues, two negative eigenvalues, and one zero eigenvalue.

Congruent symmetric matrices
Two symmetric matrices A and B are said to be congruent if there exists an orthogonal matrix Q for which A = QTBQ.

Illustration
◼ Two congruent symmetric matrices

A = {{3, -−1}, {-−1, 3}}; B = {{2, 0}, {0, 4}};

Solve[{{a, b}, {b, c}}.B.{{a, b}, {b, c}} == {{3, -−1}, {-−1, 3}}];

1

1

1

Q = {{a, b}, {b, c}} /∕. a → -− , c →

, b → -− 

2

2

2

62 | The Linear Algebra Survival Guide

1

1

1

1

-− , -− , -− ,



2

2

2

2

Q.Transpose[Q] ⩵ IdentityMatrix[2] True

A ⩵ Transpose[Q].B.Q True

◼ Congruence and quadratic forms

The previous calculations show that following two matrices are congruent:

3 -−1

20

A=

and B =

-−1 3

04

◼ Plotting q[x, y] and q[u, v]

q[x_, y_] := {x, y}.A.{x, y}

plot1 = Plot3D[q[x, y], {x, -−5, 5}, {y, -−5, 5}]

q[u_, v_] := {u, v}.B.{u, v} plot2 = Plot3D[q[u, v], {u, -−5, 5}, {v, -−5, 5}]

The Linear Algebra Survival Guide | 63

Conjugate transpose
The conjugate transpose of a complex matrix is the result of transposing the matrix and replacing its elements by their conjugates.
Illustration
◼ The conjugate transpose of a 2-by-2 complex matrix
MatrixForm[A = {{3 + I, -−5}, {-−I, 4 -− I}}] 3 + ⅈ -−5 -−ⅈ 4 -− ⅈ
MatrixForm[ConjugateTranspose[A]] 3 -− ⅈ ⅈ -−5 4 + ⅈ ◼ The conjugate transpose of a 3-by-3 real matrix
MatrixForm[A = RandomReal[{0, 9}, {3, 3}]];
5.77847 3.40248 8.44687 A = 5.34368 6.38133 1.38616
0.838548 0.3028 8.92482 MatrixForm[ConjugateTranspose[A] ⩵ Transpose[A]] True

Consistent linear system
A linear system is consistent if it has a solution. A consistent system can have either one solution or infinitely many solutions. In the latter case, the system is said to be underdetermined.

Illustration
◼ A consistent linear system in two equations and two variables

system = {3 x -− 5 y ⩵ 1, x + y ⩵ 10};

solution = Solve[system]

51

29

x → , y → 

8

8

system /∕. solution {{True, True}}

64 | The Linear Algebra Survival Guide

◼ A consistent linear system in two equations and three variables system = {3 x -− 5 y + z ⩵ 1, x + y ⩵ 10}; solution = Solve[system] {{y → 10 -− x, z → 51 -− 8 x}}

Simplify[system /∕. solution] {{True, True}}

◼ A consistent linear system in three equations and two variables system = {3 x -− 5 y ⩵ 1, x + y ⩵ 10, 4 x -− 4 y ⩵ 11};

solution = Solve[system]

51

29

x → , y → 

8

8

system /∕. solution

{{True, True, True}}

◼ A consistent underdetermined linear system Clear[x, y, z, b]

A = {{1, 1, 1}, {1, 1, 3}}; b = {1, 4};

LinearSolve[A, b]

1

3

-− , 0, 

2

2

Reduce[{x + y + z ⩵ 1, x + y + 3 z ⩵ 4}, {x, y}]

3

1

z ⩵ && y ⩵ -− -− x

2

2

As we can see, the LinearSolve command produces one solution, whereas the Reduce command produces all solutions.

Manipulation
◼ Consistent and non-consistent linear systems Clear[x, y, a, b] system = {x + a y ⩵ 1, x -− b y ⩵ 2};

The Linear Algebra Survival Guide | 65
Manipulate[Evaluate[Reduce[system, {x, y}]], {a, -−5, 5, 1}, {b, 0, 10, 1}]
a b
1 x ⩵ 2 && y ⩵ (-−1 + x)
5
We can combine Manipulate and Reduce to explore the consistency of linear systems. If we let a = - 5 and b = 0, the manipulation shows that the resulting system is consistent. Other assignments produce inconsistent systems. For example, the assignment a = b = 0 produces an inconsistent system.
Contraction along a coordinate axis
For any real number 0 < s < 1, a left-multiplication of a vector v in ℝ2 by the matrix MatrixForm[A = {{s, 0}, {0, 1}}]
10
2
01 is a contraction along the x-axis. A left-multiplication by the matrix MatrixForm[B = {{1, 0}, {0, s}}]
10 01
2
is a contraction along the y-axis.
Illustration
◼ Contractions along the coordinate axes Clear[A, B, s] MatrixForm[A = {{s, 0}, {0, 1}}]
s0 01

66 | The Linear Algebra Survival Guide

MatrixForm[B = {{1, 0}, {0, s}}]
10 0s

1 v = {3, 5}; s = ;
2
{A.v, B.v}

3

5

 , 5, 3, 

2

2

Coordinate conversion matrix
Let v be a vector in an n-dimensional real or complex vector space V, let [v]B1 be the coordinate vector of v in a basis B1, and let [v]B1 be the coordinate vector of v in a basis B2. Then there exists an invertible matrix P for which P[v]B1 = [v]B2 and P-−1[v]B2 = P[v]B1 . The matrix P is called a coordinate conversion matrix.

Illustration
◼ A 2-by-2 coordinate conversion matrix from the basis B1 to the basis B2 for ℝ2 Clear[P, v, a, b]

B1 = {{1, 0}, {0, 1}}; B2 = {{2, 1}, {1, 3}}; v = vB1 = {5, -−3};

Reduce[{5, -−3} ⩵ a {2, 1} + b {1, 3}, {a, b}]

18

11

a ⩵ && b ⩵ -−

5

5

18

-−11

vB2 = {a, b} /∕. a → , b →



5

5

18 11  , -− 
55

Reduce[{1, 0} ⩵ a {2, 1} + b {1, 3}, {a, b}]

3

1

a ⩵ && b ⩵ -−

5

5

Reduce[{0, 1} ⩵ a {2, 1} + b {1, 3}, {a, b}]

1

2

a ⩵ -− && b ⩵

5

5

The Linear Algebra Survival Guide | 67

3 -−1 -−1 2

P =  , ,  , ;

55

55

P.{5, -−3}

18 11  , -− 
55

18 11 Inverse[P]. , -− 
55
{5, -−3}

◼ A coordinate conversion matrix for two bases for ℝ3 Clear[a, b, c]

B1 = {{1, 2, 3}, {0, 1, 0}, {1, 0, 1}}; B2 = {{0, 1, 1}, {1, 1, 0}, {1, 0, 1}};
The columns of the coordinate conversion matrix from B1 to B2 are the coordinate vectors of the B1 basis vectors in the basis B2. Reduce[{1, 2, 3} ⩵ a {0, 1, 1} + b {1, 1, 0} + c {1, 0, 1}, {a, b, c}]
a ⩵ 2 && b ⩵ 0 && c ⩵ 1

Reduce[{0, 1, 0} ⩵ a {0, 1, 1} + b {1, 1, 0} + c {1, 0, 1}, {a, b, c}]

1

1

1

a ⩵ && b ⩵ && c ⩵ -−

2

2

2

Reduce[{1, 0, 1} ⩵ a {0, 1, 1} + b {1, 1, 0} + c {1, 0, 1}, {a, b, c}]

a ⩵ 0 && b ⩵ 0 && c ⩵ 1

1 1 -−1 P = Transpose{2, 0, 1},  , , , {0, 0, 1};
22 2
v = {5, 4, -−2};

Reduce[v == a {1, 2, 3} + b {0, 1, 0} + c {1, 0, 1}, {a, b, c}]

7

17

a ⩵ -− && b ⩵ 11 && c ⩵

2

2

Reduce[v == d {0, 1, 1} + e {1, 1, 0} + f {1, 0, 1}, {d, e, f}]

3

11

1

d ⩵ -− && e ⩵ && f ⩵ -−

2

2

2

68 | The Linear Algebra Survival Guide

-−7

17

3 11 1

P. , 11,  == -− , , -− 

2

2

22 2

True

Coordinate system
Vectors in ℝ2 and ℝ3 are often represented as arrows in a Cartesian coordinate system. The system consists of two perpendicular lines that cross at a point {0, 0} referred to as the origin of the system. The points along the horizontal axis (the x-axis) are labeled with the real numbers in their natural ordering, with the number 0 at the origin, positive real numbers to the right, and negative real numbers to the left of the origin. The points along the vertical axis (the y-axis) are also labeled with the real numbers in their natural ordering, with the number 0 at the origin. Positive real numbers are placed above the x-axis and negative ones below.
Illustration
◼ A Cartesian coordinate system for ℝ2
Plot[{}, {x, -−10, 10}, Axes → True]
1.0
0.5

-−10

-−5

-−0.5

5

10

-−1.0 The idea of a graph in a Cartesian coordinate system can be extended to ℝ3. ◼ A Cartesian coordinate system for ℝ3

The Linear Algebra Survival Guide | 69 Plot3D[{{}}, {x, -−10, 10}, {y, -−10, 10}, Axes → True]

1.0 0.5 0.0 -−0.5 -−1.0
-−10 -−5 0

10 5
0
-−5 5
10 -−10

Coordinate vector
The coordinate vector vB in a basis B = {b1, ..., bn} of a vector v in the standard basis of an n-dimensional real or complex vector space V is the vector {a1, ..., an} in ℝn or ℂn with the coordinates a1, ..., an for which v = a1 b1 + ⋯ + an bn.

Illustration
◼ A coordinate vector in ℝ2 Quit[]

v = {3, -−5}; B = {{1, 2}, {-−3, 4}};

Reduce[v ⩵ a1 B[[1]] + a2 B[[2]], {a1, a2}]

3

11

a1 ⩵ -− && a2 ⩵ -−

10

10

-−3 -−11

vB =  ,

;

10 10

The vector vB is the coordinate vector of v in the basis B, because the elements of vB are the coefficients, a1 and a2, needed to express v as a linear combination of the basis vectors b1 and b2.

◼ A coordinate vector in ℝ3

v = {3, 4, 5}; B = {{1, 1, 1}, {1, -−1, 1}, {0, 0, 1}};

Reduce[v ⩵ a1 B[[1]] + a2 B[[2]] + a3 B[[3]], {a1, a2, a3}]

7

1

a1 ⩵ && a2 ⩵ -− && a3 ⩵ 2

2

2

70 | The Linear Algebra Survival Guide

7 -−1 vB =  , , 2;
22

Correlation coefficient

The (Pearson) correlation coefficient combines covariances and standard deviations into a single formula that measures the degree to which two vectors in ℝn are correlated.

Dot[x, y]

Covariance[x, y] /∕ (StandardDeviation[x] StandardDeviation[y]) =

(1)

Norm[x] Norm[y]

Illustration
◼ The correlation coefficient of two data sets as a cosine of an angle
x = {2, 6, 3, 1, 8, 9}; y = {6, 2, 0, -−3, 9, 2}; mx = Mean[x]; my = Mean[y];
1 pcx = Table[(x[[i]] -− mx), {i, 1, 6}];
6 1 pcy = Table[(y[[i]] -− my), {i, 1, 6}]; 6 pcv[x, y] = 6 Total[pcx pcy]
lhs = (1 /∕ 6) (1 /∕ 5) pcv[x, y] /∕ (StandardDeviation[pcx] StandardDeviation[pcy])
rhs = Dot[pcx, pcy] /∕ (Norm[pcx] Norm[pcy])
lhs ⩵ rhs

Correlation matrix
A correlation matrix is an m-by-m matrix whose elements are the pairwise correlation coefficients of m vectors in ℝn.
Illustration
◼ The correlation matrix of three data sets
x = {2, 6, 3, 1, 8}; y = {6, 2, 0, -−3, 9}; z = {1, 2, 1, 2, -−5};
lhs = MatrixForm[N[Correlation[Transpose[{x, y, z}]]]]
rhs = MatrixForm[N[ {{Correlation[x, x], Correlation[x, y], Correlation[x, z]}, {Correlation[y, x], Correlation[y, y], Correlation[y, z]}, {Correlation[z, x], Correlation[z, y], Correlation[z, z]}}]]

The Linear Algebra Survival Guide | 71

lhs ⩵ rhs

Cosine of an angle

The cosine of an angle between two vectors u and v in a vector space V, equipped with an inner product 〈u, v〉, is the scalar defined by the formula

〈u, v〉

cosine[u, v] =

(1)

〈u, u〉 〈v, v〉

Illustration
The dot product and Euclidean norm of a vector can be used to find the cosine of the angle between two vectors. ◼ The cosine of the angle between two vectors in ℝ2 u = {1, 2}; v = {3, 4}; cosine[u_, v_] := (Dot[u, v]) /∕ (Norm[u] Norm[v]) angle[u, v] = N[ArcCos[cosine[u, v]]] 0.179853 ◼ The cosine of the angle between two vectors in ℝ3 u = {1, 2, 3}; v = {4, 5, 6}; cosine = (Dot[u, v]) /∕ (Norm[u] Norm[v])

16 2
11
7 N[cosine] 0.974632

The dot product, norm, and cosine are related by the identity

Dot[x, y] == Norm[x] Norm[y] cosine[x, y]

(2)

◼ The dot product, Euclidean norm, and cosine identity

x = {1, 2}; y = {3, 4};

N[cos[x, y] = (Dot[x, y]) /∕ (Norm[x] Norm[y])] 0.98387

72 | The Linear Algebra Survival Guide
Dot[x, y] == Norm[x] Norm[y] cos[x, y] True
Covariance
The covariance of two vectors in ℝn measures how the vectors vary together. It is based on the dot product of the meandeviation form of the vectors.
Illustration
◼ Dot products and the covariance of two data sets
x = {2, 6, 3, 1, 8, 9}; y = {6, 2, 0, -−3, 9, 2}; mx = Mean[x]; my = Mean[y];
1 pcx = Table[(x[[i]] -− mx), {i, 1, 6}];
6 1 pcy = Table[(y[[i]] -− my), {i, 1, 6}]; 6 pcv[x, y] = 6 Total[pcx pcy];
1 scx = Table[(x[[i]] -− mx), {i, 1, 6}];
5 1 scy = Table[(y[[i]] -− my), {i, 1, 6}]; 5 scv[x, y] = 5 Dot[scx, scy] 101 15 scv[x, y] ⩵ Covariance[x, y] True
Covariance matrix
The (sample) covariance matrix of different vectors in ℝn is the matrix whose elements are the pairwise covariances of the vectors. The diagonal elements of the matrix are (sample) variances of the individual sets of data.
Illustration
◼ Variances and covariances

The Linear Algebra Survival Guide | 73

x = {2, 6, 3, 1, 8}; y = {6, 2, 0, -−3, 9}; z = {1, 2, 1, 2, -−5};

{Variance[x], Variance[y], Variance[z]}

17 227 87 , , 
2 10 10

lhs = MatrixForm[Covariance[Transpose[{x, y, z}]]]

17 2 37 4 25
-−
4

37 4 227 10 219
-−
20

25
-−
4 219
-−
20 87 10

rhs = MatrixForm[ {{Variance[x], Covariance[x, y], Covariance[x, z]}, {Covariance[y, x], Variance[y], Covariance[y, z]}, {Covariance[z, x], Covariance[z, y], Variance[z]}}]

17 2 37 4 25
-−
4

37 4 227 10 219
-−
20

25
-−
4 219
-−
20 87 10

lhs ⩵ rhs True

Cramer’s rule

If Av = b is a linear system and A is an n-by-n invertible matrix, then Cramer’s rule says that for each 1 ≤ i ≤ n,

Det[A[vi /∕ b]]

vi =

(1)

Det[A]

where Avi  b is the matrix obtained by replacing the ith column of A by the vector b.

Illustration
Quit[] ◼ Using Cramer’s rule to solve a linear system system = {x + y -− z ⩵ 4, 2 x -− y + 2 z ⩵ 18, x -− y + z ⩵ 5}; variables = {x, y, z};

74 | The Linear Algebra Survival Guide

Solve[system, variables]

9

17

x → , y → 8, z → 

2

2

MatrixForm[cA = Normal[CoefficientArrays[system, {x, y, z}]][[2]]]

1 1 -−1 2 -−1 2 1 -−1 1

A[x /∕ b] = {{4, 1, -−1}, {18, -−1, 2}, {5, -−1, 1}};

A[y /∕ b] = {{1, 4, -−1}, {2, 18, 2}, {1, 5, 1}};

A[z /∕ b] = {{1, 1, 4}, {2, -−1, 18}, {1, -−1, 5}};

Det[A[x /∕ b]]

Det[A[y /∕ b]]

Det[A[z /∕ b]]

solution = x =

, y=

, z=



Det[cA]

Det[cA]

Det[cA]

9

17

 , 8, 

2

2

This shows that the solution found using Cramer’s rule is the same as that found with Solve.

Cross product
Although many geometric concepts in ℝ3 are the obvious extensions of their analogues in ℝ2, there exist exceptions. One such is the construction of a vector that is orthogonal to two given vectors. Only in ℝ3 can we define an operation on two vectors known as the vector cross product of the two vectors. The built-in Mathematica Cross function computes vector cross products. In ℝ3, any two nonzero linearly independent vectors u and v can be combined to create a third vector w which is orthogonal to both u and v.
Illustration
◼ A vector cross product
u = {1, 2, 3}; v = {-−3, 0, 1};
w = Cross[u, v] {2, -−10, 6}
Dot[u, w] ⩵ Dot[v, w] ⩵ 0 True

The Linear Algebra Survival Guide | 75
◼ Graphic representation of a vector cross product
Graphics3D[{Arrow[{{0, 0, 0}, {1, 2, 3}}], Arrow[{{0, 0, 0}, {-−3, 0, 1}}], Arrow[{{0, 0, 0}, {2, -−10, 6}}]}, Axes → True] 0
-−5
-−160 4 2 0 -−2 0 2
Various geometric constructions are based on the cross product. For a meaningful construction, the two given vectors need to be linearly independent nonzero vectors. ◼ A vector cross product Any two nonzero linearly independent vectors u and v in ℝ3 determine a vector w that is orthogonal to both u and v in the Euclidean inner product (dot product). u = {1, 2, 3}; v = {4, 5, 6};
w = Cross[u, v] {-−3, 6, -−3}
Dot[w, u] ⩵ Dot[w, v] ⩵ 0 True
The cross product can be calculated using the determinant formula for 1-by-3 vectors, together with the symbolic vectors i, j, and k. ◼ Using determinants to calculate a vector cross product
MatrixForm[A = {{i, j, k}, {1, 2, 3}, {4, 5, 6}}] ijk 123 456
Det[A] -−3 i + 6 j -− 3 k

76 | The Linear Algebra Survival Guide
i = {1, 0, 0}; j = {0, 1, 0}; k = {0, 0, 1}; Cross[x, y] ⩵ -−3 i + 6 j -− 3 k x ⨯ y ⩵ {-−3, 6, -−3} The absolute values of the norm of the cross product represents the area of the parallelogram determined by the vectors x and y. ◼ Areas of parallelograms and vector cross products u = {1, 2, 3}; v = {4, 5, 6}; w = Cross[u, v]; parallelogram = Abs[Norm[w]] 36 The area of one of the triangles obtained from the parallelogram by dividing it using one of its diagonals is half the area of the parallelogram. ◼ Areas of triangles and vector cross products u = {1, -−2, 3}; v = {4, 5, 1}; w = Cross[u, v];
1 triangle = Abs[Norm[w]]
2 579 2
Manipulation
◼ The vector cross product of two nonzero vectors in ℝ3 Manipulate[Cross[{a, 2, 3}, {4, b, 6}], {a, -−3, 3}, {b, -−2, 2}]
a b
{13.65, 8.4, -−8.33}

The Linear Algebra Survival Guide | 77
We use Manipulate and Cross to explore the cross product of two nonzero vectors. As expected, the resulting vector is (approximately) orthogonal to both the vectors {0.6, 2, 3} and {4, - 0.55, 6}: Chop[Dot[{0.6, 2, 3}, {13.65`, 8.399999999999999`, -−8.33`}]] 0 Chop[Dot[{4, -−0.55, 6}, {13.65`, 8.399999999999999`, -−8.33`}]] 0

78 | The Linear Algebra Survival Guide

D

Defective matrix
An n-by-n matrix is defective if it does not have a set of n linearly independent eigenvectors. Defective matrices are not diagonalizable.

Illustration
◼ A 4-by-4 defective matrix
MatrixForm[A = Normal[SparseArray[{{2, 3} → 1, {3, 2} → 0}, {4, 4}]]]
0000 0010 0000 0000

Eigenvectors[A] {{0, 0, 0, 1}, {0, 1, 0, 0}, {1, 0, 0, 0}, {0, 0, 0, 0}}

The calculation shows that the 4-by-4 matrix A has a maximum of three linearly independent eigenvectors. It is therefore a defective matrix. An n-by-n real matrix may not have n linearly independent real eigenvectors and may therefore be considered to be defective as a real matrix. ◼ A real 2-by-2 matrix (defective as a real matrix)
MatrixForm[A = {{Cos[π /∕ 3], Sin[π /∕ 3]}, {-−Sin[π /∕ 3], Cos[π /∕ 3]}}]

1

3

2

2

3

1

-−

2

2

N[Eigenvectors[A]]

-−1.4803 × 10-−16 -− 1. ⅈ, 1., 2.22045 × 10-−16 + 1. ⅈ, 1.

The calculations show that the real matrix A has no real eigenvectors. Hence it is defective (as a real matrix).

The Linear Algebra Survival Guide | 79
Determinant
The determinant of a square matrix is a scalar associated with the matrix. It is defined by induction on the size of the matrix.
Illustration
◼ The determinant of a 1-by-1 matrix MatrixForm[A = {{a}}] (a)
Det[A] a
◼ The determinant of a 2-by-2 matrix MatrixForm[A = {{a, b}, {c, d}}]
ab cd TraditionalForm[Det[A]] a d -− b c ◼ The determinant of a 3-by-3 matrix MatrixForm[A = {{a, b, c}, {d, e, f}, {g, h, i}}] abc def ghi Det[A] -−c e g + b f g + c d h -− a f h -− b d i + a e i
Various formulas for calculating determinants exist. Here is an example of the Laplace expansion of the determinant of a 3by-3 matrix. ◼ The determinant of a matrix calculated by an expansion along the first row of the matrix MatrixForm[A = {{a, b, c}, {d, e, f}, {g, h, i}}]
abc def ghi The determinant of 3-by-3 matrix A can be calculated as a linear combination of the first row of A and the determinants of three 2-by-2 submatrices of A. A11 = {{e, f}, {h, i}};
A12 = {{d, f}, {g, i}};

80 | The Linear Algebra Survival Guide
A13 = {{d, e}, {g, h}}; Expand[Det[A] ⩵ a Det[A11] -− b Det[A12] + c Det[A13]] True
Manipulation
◼ Determinants of 3-by-3 matrices Manipulate[Det[{{a, b, c}, {1, 2, 3}, {4, 5, 6}}], {a, -−2, 2, 1}, {b, -−2, 2, 1}, {c, -−2, 2, 1}]
a b c
0
We can combine Manipulate and Det to explore the determinants of matrices. If we assign the values a = b = c = - 2, for example, the manipulation shows that the determinant of the resulting matrix is zero. Other assignments to a, b, and c such as a = - 2, b = - 1, and c = - 2 produce a matrix with a nonzero determinant:
a b c
6
Diagonal
See Diagonal of a matrix, Jordan block, subdiagonal, superdiagonal
Diagonal decomposition
Eigenvalues and eigenvectors are needed for the diagonal decomposition of a matrix A into a product of the form P dM P-−1 consisting of an invertible matrix P whose columns are eigenvectors of A and a diagonal matrix dM whose diagonal entries are eigenvalues of A. The decomposition of an n-by-n real matrix requires n linearly independent eigenvectors.

The Linear Algebra Survival Guide | 81

P dM P-−1 consisting of an invertible matrix P whose columns are eigenvectors of A and a diagonal matrix dM whose diagonal entries are eigenvalues of A. The decomposition of an n-by-n real matrix requires n linearly independent eigenvectors.

Illustration

Eigenvectors and eigenvalues are the building blocks of diagonal decompositions of real matrices. Suppose we would like to rewrite a matrix A as a product

P.DiagonalMatrix[Eigenvalues[A]].Inverse[P]

(1)

then the diagonal matrix DiagonalMatrix[Eigenvalues[A]] must consist of the eigenvalues of A and the columns of P must be associated eigenvectors.
◼ A diagonal decomposition of a 3-by-3 real matrix

MatrixForm[A = {{2, 1, 1}, {4, 1, 7}, {5, 3, 0}}]
211 417 530

evalues = Eigenvalues[A] {7, -−4, 0}

MatrixForm[dM = DiagonalMatrix[evalues]] 700 0 -−4 0 000
MatrixForm[evectors = Eigenvectors[A]] 132 1 -−19 13 -−3 5 1
Mathematica outputs the eigenvectors of A as row vectors. In order to form a matrix whose columns are eigenvectors, we must transpose them. MatrixForm[A == Transpose[evectors].dM.Inverse[Transpose[evectors]]] True
The matrices
MatrixForm[P = Transpose[evectors]] and
MatrixForm[dM = DiagonalMatrix[evalues]]

82 | The Linear Algebra Survival Guide

yield a diagonal decomposition of the matrix A. The matrix P is not unique. Different choices of eigenvectors produce different decompositions.

Manipulation
All real 3-by-3 matrices have at least one real eigenvalue since their characteristic polynomials are real polynomials of odd degree and real polynomials of odd degree cut the x-axis at least once. The point of intersection of the graph of the polynomial and of the x-axis corresponds to a real eigenvalue of the matrix. ◼ Using Manipulate to explore eigenvalues
The matrix
MatrixForm[A = {{1, 0, 5}, {6, 2, 0}, {1, 0, 3}}]
105 620 103
has three real eigenvalues.
Eigenvalues[A]

2 + 6 , 2, 2 -− 6  We can use the Plot function to visualize these eigenvalues as the roots of the characteristic polynomials of A. cpA = CharacteristicPolynomial[A, t] -−4 -− 6 t + 6 t2 -− t3

Plot[cpA, {t, -−5, 5}]
300 250 200 150 100
50

-−4

-−2

2

4

What happens if we replace the third row of A by {a, 0, 3}, with a ranging over a wider interval of scalars?

A = {{1, 0, 5}, {6, 2, 0}, {a, 0, 3}} {{1, 0, 5}, {6, 2, 0}, {a, 0, 3}}

Manipulate[Eigenvalues[{{1, 0, 5}, {6, 2, 0}, {a, 0, 3}}], {a, -−5, 5}]

The Linear Algebra Survival Guide | 83
a
2 1 + ⅈ 6 , 2 1 -− ⅈ 6 , 2
We can combine Manipulate and Eigenvalues to explore the nature of the eigenvalues of matrices. The manipulation shows that for negative values of a, some of the eigenvalues of the real matrix A are not real. Therefore the resulting matrix is not diagonalizable, although it has three distinct (complex) eigenvalues.
Diagonal matrix
A diagonal matrix A is a square array whose elements A[[i,j]] in the ith row and jth column are zero if i≠ j. For some applications it is convenient to extend this definition to rectangular matrices. In that case, the matrices are padded with either zero rows and/or zero columns and are sometimes called generalized diagonal matrices.
Illustration
◼ A 5-by-5 diagonal matrix MatrixForm[DiagonalMatrix[{1, 2, 3, 4, 5}]]
10000 02000 00300 00040 00005 ◼ A generalized diagonal matrix obtained by appending a row of zeros MatrixForm[Append[DiagonalMatrix[{1, 2, 3, 4, 5}], {0, 0, 0, 0, 0}]] 10000 02000 00300 00040 00005 00000 ◼ A generalized diagonal matrix obtained by appending a column of zeros

84 | The Linear Algebra Survival Guide

MatrixForm[Transpose[Append[DiagonalMatrix[{1, 2, 3, 4, 5}], {0, 0, 0, 0, 0}]]]
100000 020000 003000 000400 000050
Diagonal matrices can be created using the SparseArray function by specifying the nonzero elements.
◼ A 4-by-4 diagonal matrix
MatrixForm[Normal[SparseArray[{{1, 1} → 5, {2, 2} → 2, {3, 3} → 5}, {4, 4}]]]
5000 0200 0050 0000
◼ A 4-by-5 diagonal matrix
MatrixForm[Normal[SparseArray[{{1, 1} → 5, {2, 2} → 2, {3, 3} → 5, {4, 4} → 6}, {4, 5}]]]
50000 02000 00500 00060

Diagonal of a matrix
The diagonal of an m-by-n matrix A is the list of all elements A[[i,i]] of A for i from 1 to m.

Illustration

◼ Diagonal of a 4-by-6 matrix

A = RandomInteger[{0, 9}, {4, 6}];

280087

032441

A=

;

107820

587091

Diagonal[A]

{2, 3, 7, 0}

◼ Diagonal of a 4-by-4 matrix

The Linear Algebra Survival Guide | 85

8893

4067

A=

;

8385

8645

diagonalA = {A[[1, 1]], A[[2, 2]], A[[3, 3]], A[[4, 4]]}

{8, 0, 8, 5}

diagonalA ⩵ Diagonal[A] True

◼ Diagonal of a 4-by-5 matrix A = RandomInteger[{0, 9}, {4, 5}];

14118

91993

A=

;

22726

05619

diagonalA = {A[[1, 1]], A[[2, 2]], A[[3, 3]], A[[4, 4]]}

{1, 1, 7, 1}

The superdiagonal of an m-by-n matrix A is the list of all elements A[[i,i+1]] for i from 1 to (m - 1). ◼ The superdiagonal of a 4-by-6 matrix

280087

032441

A=

;

107820

587091

Diagonal[A, 1]

{8, 2, 8, 9}

The subdiagonal of an m-by-n matrix A is the list of all elements A[[i+1,i]] for i from 2 to m. ◼ The subdiagonal of a 4-by-6 matrix

280087

032441

A=

;

107820

587091

Diagonal[A, -−1]

{0, 0, 7}

86 | The Linear Algebra Survival Guide
Difference equation
If the vectors in a list {v0, v1, v2, ..., vn, ...} are connected by a matrix A for which vn+1 = Avn for n = 0, 1, 2, ..., then the equation vn+1 = Avn is called a linear difference equation.
Illustration
◼ A difference equation based on a 2-by-2 matrix MatrixForm[A = {{0.75, 0.5}, {0.25, 0.5}}]
0.75 0.5 0.25 0.5 v0 = {100 000, 200 000} {100 000, 200 000} The list consisting of the first three elements of the list {v0, v1, v2, ..., vn, ...} is {v0, v1 = A.v0, v2 = A.v1} {{100 000, 200 000}, {175 000., 125 000.}, {193 750., 106 250.}}
Dimension of a vector space
A vector space is finite-dimensional if it has a basis consisting of a finite number of basis vectors. Since all bases of a finitedimensional vector space have the same number of elements, this number is defined to be the dimension of the space.
Illustration
◼ A two-dimensional vector space The space ℝ2 of all pairs of real numbers {a, b} is a two-dimensional vector space. The sets B1 = {e1 = {1, 0}, e2 = {0, 1}} B2 = {b1 = {3, -−4}, b2 = {1, 1}} are two bases for the same space. ◼ A one-dimensional vector space The space ℂ of all complex numbers is a one-dimensional complex vector space. The set ℂ = {1} {1} is a basis for ℂ since every complex number z is a multiple of 1.

The Linear Algebra Survival Guide | 87

◼ A four-dimensional vector space

The space ℝ[t,3] of real polynomials of degree 3 or less is a four-dimensional vector space since the set

B = 1, t, t2, t3 is a basis for the space.

◼ A four-dimensional vector space

A = RandomInteger[{0, 9}, {4, 5}];

71975

29829

A=

;

06406

47775

B = RowReduce[A];

1 0 0 0 -−4

0 1 0 0 -−1

B=

;

0010 3

0001 1

The first four columns of the matrix B are the pivot columns of the matrix A. They therefore form a basis for the column space of A. We can use the Length function to calculate its dimension.

Length[B]

4

Dimensions of a matrix
The numbers of rows and columns of a matrix, in that order, are called the dimensions of the matrix.
Illustration
◼ A matrix of dimensions {3, 4} A = RandomInteger[{0, 9}, {3, 4}];
0722 A= 5 7 8 5 ;
9471 Dimensions[A]
{3, 4} ◼ A matrix of dimensions {4, 3} A = RandomInteger[{0, 9}, {4, 3}];

88 | The Linear Algebra Survival Guide

430

347

A=

;

615

352

Dimensions[A]

{4, 3}

◼ Dimensions of a square matrix

A = RandomInteger[{0, 9}, {4, 4}];

3456

0431

A=

;

3058

9429

Dimensions[A]

{4, 4}

Dirac matrix
The Dirac matrices are 4-by-4 matrices arising in quantum electrodynamics. They are Hermitian and unitary.
Illustration
◼ The 4-by-4 Dirac matrices
MatrixForm[I4 = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, 1}}]
1000 0100 0010 0001
MatrixForm[σ1 = {{0, 1, 0, 0}, {1, 0, 0, 0}, {0, 0, 0, 1}, {0, 0, 1, 0}}]
0100 1000 0001 0010
MatrixForm[σ2 = {{0, -−ⅈ, 0, 0}, {ⅈ, 0, 0, 0}, {0, 0, 0, -−ⅈ}, {0, 0, ⅈ, 0}}]
0 -−ⅈ 0 0 ⅈ 0 0 0 0 0 0 -−ⅈ 0 0 ⅈ 0

The Linear Algebra Survival Guide | 89
MatrixForm[σ3 = {{1, 0, 0, 0}, {0, -−1, 0, 0}, {0, 0, 1, 0}, {0, 0, 0, -−1}}] 1000 0 -−1 0 0 0010 0 0 0 -−1
MatrixForm[ρ1 = {{0, 0, 1, 0}, {0, 0, 0, 1}, {1, 0, 0, 0}, {0, 1, 0, 0}}] 0010 0001 1000 0100
MatrixForm[ρ2 = {{0, 0, -−ⅈ, 0}, {0, 0, 0, -−ⅈ}, {ⅈ, 0, 0, 0}, {0, ⅈ, 0, 0}}] 0 0 -−ⅈ 0 0 0 0 -−ⅈ ⅈ 0 0 0 0 ⅈ 0 0
MatrixForm[ρ3 = {{1, 0, 0, 0}, {0, 1, 0, 0}, {0, 0, -−1, 0}, {0, 0, 0, -−1}}] 10 0 0 01 0 0 0 0 -−1 0 0 0 0 -−1
{HermitianMatrixQ[σ1], HermitianMatrixQ[ρ3]} {True, True}
{UnitaryMatrixQ[σ1], UnitaryMatrixQ[ρ3]} {True, True}
Direct sum of vector spaces
The zero subspaces are useful for the definition of direct sums of subspaces. If two subspaces U and V of a vector space W are disjoint, in other words, if they share only the zero vector of the space, and if BU is a basis for U and BV is a basis for V, then every vector w in W can be written as a unique sum u + v, with u in BU and v in BV . The union of U and V, in that order, is called the direct sum of U and V and is written as U ⊕ V. The direct sum symbol ⊕ is produced by typing Esc c+ Esc.
Illustration
◼ A direct sum of two subspaces of ℝ4 If B1 and B2 are the two bases

90 | The Linear Algebra Survival Guide
B1 = {{1, 0, 0, 0}, {0, 1, 0, 0}}; B2 = {{0, 0, 1, 0}}; of subspaces of ℝ4 and V is the subspace of all vectors of the form {a, b, c, 0}, then W = span[B1] ⊕ span[B2] : w = {a, b, c, 0} == a {1, 0, 0, 0} + b {0, 1, 0, 0} + c {0, 0, 1, 0} True ◼ The direct sums of four vector spaces generated by the 3-by-5 matrix A = {{3, 1, 0, 2, 4}, {1, 1, 0, 0, 2}, {5, 2, 0, 3, 7}}; Dimensions[A] {3, 5} ◼ The coordinate space ℝ5 as a direct sum of the null space and the row space of a matrix A nsA = NullSpace[A] {{-−1, -−1, 0, 0, 1}, {-−1, 1, 0, 1, 0}, {0, 0, 1, 0, 0}}
rsA = RowReduce[A] {{1, 0, 0, 1, 1}, {0, 1, 0, -−1, 1}, {0, 0, 0, 0, 0}} The null space and the row space are subspaces of ℝ5 with dimensions 3 and 2. The spaces are disjoint and the sum of their dimensions is therefore 5. Solve[nsA[[1]] == a rsA[[1]] + b rsA[[2]], {a, b}] {}
Solve[nsA[[2]] == a rsA[[1]] + b rsA[[2]], {a, b}] {}
Solve[nsA[[3]] == a rsA[[1]] + b rsA[[2]], {a, b}] {}
Solve[rsA[[1]] == a nsA[[1]] + b nsA[[2]] + c nsA[[3]], {a, b, c}] {}
Solve[rsA[[2]] == a nsA[[1]] + b nsA[[2]] + c nsA[[3]], {a, b, c}] {}

The Linear Algebra Survival Guide | 91

The union of nsA and rsA forms a basis for ℝ5. This is expressed by saying that ℝ5 is a direct sum of the null and row spaces. The notation NullSpace[A] ⊕ RowSpace[A] ⩵ ℝ5 expresses the fact that the direct sum of the two disjoint subspaces is all of ℝ5. ◼ The coordinate space ℝ3 as a direct sum of the left null space and the column space of A
lnsA = NullSpace[Transpose[A]]
{{-−3, -−1, 2}}

csA = RowReduce[Transpose[A]]

3

1

1, 0, , 0, 1, , {0, 0, 0}, {0, 0, 0}, {0, 0, 0}

2

2

The left null space and the column space are subspaces of ℝ3 with dimensions 1 and 2. The spaces are disjoint and the sum of their dimensions 3.

Solve[lnsA[[1]] == a csA[[1]] + b csA[[2]], {a, b}]

{}

Solve[csA[[1]] == a lnsA[[1]], a] {}

Solve[csA[[2]] == a lnsA[[1]], a] {}
Hence we can form the direct sum leftnullSpace[A]⊕ columnSpace[A] of the left null space and the column space to build ℝ3. It may happen that some of the four subspaces of a matrix are zero spaces. In that case, their bases are empty and their dimensions therefore are zero. ◼ Direct sums involving zero subspaces
MatrixForm[A = {{1, 2}, {3, 4}}] 12 34
NullSpace[A] {}

MatrixForm[RowReduce[A]]
10 01