This book is meant as a textbook for undergraduate and graduate students who are willing to understand essential elements of machine learning from both a theoretical and a practical perspective. The choice of the topics in the book is made based on one criterion: whether the practical utility of a certain method justifies its theoretical elaboration for students with a typical mathematical background in engineering and other quantitative fields. As a result, not only does the book contain practically useful techniques, it also presents them in a mathematical language that is accessible to both graduate and advanced undergraduate students.