Computational Linear Algebra

Index

Overview

tl;dr

Linear algebra helps us describe and solve problems in the natural sciences, social sciences, and humanities. Typically, the problems are complicated enough that we can’t solve them by hand. Instead, we solve them computationally, but this approach creates new challenges. In this course, we will develop theory for doing linear algebra on computers and use it to solve problems from diverse fields.

The longer version:

Linear algebra is of central importance not just within mathematics itself, but in application areas such as image and signal processing, statistics, machine learning, computational finance, structural biology, network science, natural language processing… and that’s just the tip of the iceberg. In many such applications, pencil-and-paper approaches to linear algebra are not feasible, and computation is necessary. I mean, do you really want to try to do Gaussian elimination by hand on a 1,000,000 x 1,000,000 matrix? I DIDN’T THINK SO.

When linear algebra problems must be solved computationally, issues like approximation, round-off error, convergence, and efficiency matter, and traditional linear algebra algorithms — even when implemented on a computer — may fail. This course is about using linear algebra on a large scale, implementing the appropriate techniques computationally, and applying them in broad contexts.

I have on purpose made this course lean. That is to say, as compared to a typical (whatever that means) course, I prefer to cover fewer topics and special cases/details, and perhaps even do fewer problems but do them extremely meticulously. This creates an environment where we can learn carefully, and where I can spend more time helping you master the selected ideas and techniques that I think are most important.

This course is adapted from one originally developed at Macalester College by Danny Kaplan, with subsequent development by Tom Halverson, David Shuman, and Lori Ziegelmeier. Some of the course materials appearing on this website are borrowed from them or adapted from them.

Syllabus

As with all courses I teach, the syllabus for this course is a rather succinct visual syllabus which you can find here.

Course FAQ

What Are the Prerequisites?

To enroll in my course, you’ll need to have taken a standard undergraduate course in linear algebra. I consider my course to be a second course in linear algebra. It will reinforce some of the key ideas of the subject and teach you many new ones. Additionally, I strongly recommend that you gain some computer programming experience before enrolling in my course. We will be coding for every assignment/activity that we undertake. More specifically, we will program in R. Prior experience in R is not necessary, as the first unit of the course will be an introduction to (or, if applicable, review of) some R basics.

Do I expect that you remember all of linear algebra perfectly? Do I expect that you are a software engineer? No to both. But to be able to complete the basics of this course and succeed in learning new material, you’ll need the background described above.

What Even Is Computational Linear Algebra?

Here’s the deal. Linear algebra is arguably THE most important part of modern applied mathematics. When solving real world applied mathematical problems using linear algebra, doing it with a pencil and paper usually doesn’t work because the problems are too big. Do you really want to try to do Gaussian elimination by hand on a 1,000,000 x 1,000,000 matrix? I DIDN’T THINK SO. We instead use computers to solve linear algebra problems algorithmically. As soon as we are solving problems by implementing algorithms on computers, though, we have to worry about things like “what kinds of errors am I introducing into the problem?” and “how long is my program going to take to solve this problem?” In short, this course is a mash up of linear algebra, computing, and applications in various areas.

What Are We Actually Going To Do In This Course?

For a typical unit of this course, you will watch some videos, do reading, take some multiple choice questions to assess your basic understanding of key concepts, meet in groups with me, and work on a challenging and educational set of problems. The videos will often not be videos of me. Why? Learning science research suggests that if a lecture is clear, there is nothing special about whether or not it is your professor delivering it. You might be wondering, then, what is my job as a professor if not to lecture to you? Well, it’s for me to work with you in a highly individualized way, helping to address the material that you find most challenging, helping you to explore your own interests, and much more.

How Will I Be Graded?

This course is ungraded. The way I implement ungrading is that every so often, I will have you write a rather detailed self-reflection about your goals and progress in the course. None of the problem sets you complete “count” towards a final grade. I focus on giving you substantial written/verbal feedback that aids your learning. At the end of the course, I will ask you to think about your self-reflections and assessment of yourself and tell me what grade to assign you. Then I will assign you that grade. The one restriction is that I will not assign an A+ grade. Please realize that if you ever ask me for a letter of recommendation, it will be based on my own assessment of your work and not on the grade you assign yourself.

What Do I Need to Ask Myself?

As you decide whether you want to follow through with taking this course, please ask yourself the following questions.

(1) Am I ok taking a class where the professor doesn’t lecture at all during class? This means I will need to prepare substantially before class time by reading and/or watching videos.

(2) Am I ok taking a class where coding in R plays a central role? Am I ok that this means debugging code will be a thing that I often need to do? Am I willing to view that as not detracting from my educational experience, but rather, being part of my educational experience? Also, am I ok taking a class where the first week and a half will largely be spent learning/reviewing R? For instance, if you already are an experienced programmer in R, you are going to be bored, and will want to work with me to get paired up to provide support for other students.

(3) Am I ok taking a class where the professor’s fundamental goal is to create a learning community in which each student takes responsibility not only for their own learning, but for every single other student’s learning as well?

(4) Am I ok taking a class that is extremely applied? This is generally NOT a proofs-based class. We focus on understanding useful ideas in linear algebra and implementing them computationally to solve problems.

(5) Am I ok taking a class where nearly every single assignment involves collaboration? There is very little individual work in this class, which is also reflective of how a lot of applied mathematical work takes place. If you are the type of person who envisions saying to yourself “I don’t understand why I can’t work on this course by myself” then this is absolutely, 100% not the course for you. You will not be happy.

What Is the Point, Chad?

Please think carefully about if this is a course that’s right for you. I design it in a purposeful way to comport with my reading of the research literature in learning science. Based on that reading, I believe you will learn more if you can motivate yourself to invest in the course without the extrinsic measure of grades, and if you tackle difficult, open ended work. However, if you believe you are the type of person who will only feel comfortable if I lecture to you, or will force yourself to invest time in the course only in order to avoid the threat of a low grade from a professor, this is not the course for you.

R Resources
Course Notes

Here are the notes I use for this course. They are getting cleaner all the time but still may contain some errors and typos. If you see anything that needs to fixed, I would be grateful if you would contact me to let me know.

Course Unit Pages
Ungraded Midterms

Since my course is ungraded, I actually don’t give any exams. However, because problem sets are done collaboratively, I do want the students to have a chance to grapple with problems on their own, and I have two separate weeks of my course during each of which students complete what is essentially an ungraded midterm (but with substantial written feedback given). Some of the best and most interesting problems in the entire course appear here!

These exercises let them assess their own understanding, and let me know what material I should return to in order to better solidify.

The Future, and Want to Help?

For instructors reading this: I hope these resources are useful to you. There are still a few small holes in them, however. In particular, the notes still need a section on principal component analysis, and checkpoint quizzes need to be written for least squares, Eigenvalues, and principal component analysis. And finally, there is always room to add creative and challenging problems. If you yourself create any of these materials and would like to share them here, I would gratefully welcome your contributions with plenty of due credit given, of course!