following is a short interview with Succinctly
series author Alexandre
latest book, Support Vector Machines
Succinctly, was published on Monday, October 23. You can download the book here.
What should people know about support
vector machines? Why are they important?
Support vector machines (SVM) are
important because they have been successfully used in three main areas: text
categorization, image recognition, and bioinformatics. They are based on a
strong mathematical background, and they generalize well. If we see machine
learning as a discipline where you need to find the right tool to do the job,
then SVMs are very powerful tools. However, they have limitations, and they are
often seen as “black boxes,” so it is important to understand them well.
When did you first become
interested in support vector machines?
Three years ago, we had a project
at work where we needed to classify financial news in real time. The
traditional approach using keywords was giving us poor results, so a colleague
of mine who had previous experience in natural language processing suggested
that we try to use machine learning, support vector machines in particular. I
already knew about SVMs because of the machine learning courses I had taken,
but this project really made me use them. I learned a lot about how to use
them, and I became interested in understanding how and why they work. That is
when I started writing my blog svm-tutorial.com where
I try to explain the mathematics behind SVMs.
By writing this e-book, did you
learn anything new yourself?
I learned a lot because I read
many different resources explaining support vector machines. I wanted to read
the work of different authors in order to see what was easily explained, and
what was not explained in detail. For instance, how to solve an optimization
problem with constraints is often suggested as being something secondary when
in fact I think it helps the interested reader to really understand how SVMs
work. I did not find any real overview of multi-class classification either, so
I dedicated a chapter of the book to it and learned about all these different
methods. Implementing the sequential minimal optimization algorithm in Python
from the pseudocode in the original paper gave me a very good understanding of
how it works. If you have time, I recommend writing a book as a great way of
How will support vector machines
change over the next few years?
The theory behind SVMs will not
change, but maybe we will start seeing them used in other applications. In deep
learning, people are trying to incorporate SVMs into convolutional neural
networks, while others are researching variations of the original SVM.
Do you see support vector machines
as part of a larger trend in software development?
Machine learning is gaining a lot
of traction lately and is definitely becoming a big trend in software
What other books or resources on support
vector machines do you recommend?
I absolutely recommend the course
“Learning from Data,” which explains quite well why learning is possible and
has a section on SVMs: work.caltech.edu/telecourse.
For a very mathematical
introduction you can read: An
Introduction to Support Vector Machines and Other Kernel-based Learning Methods by Cristianini and Shawe-Taylor. For a friendlier approach,
Andrew Ng’s Machine Learning course on Coursera is very good for beginners.