this is the math necessary for most comp-sci majors.
specifically to machine learning, which topics in mathematics are most relative to be pursued?
what pathway should a math major take to have the most relative knowledge to pursue ML
sorry that the question came out so ambiguous.
yes, by school i mean uni.
the question may have better been asked as "what is the best route to take for a mathematics student to learn the most relevant ideas in ML?"
i've take all the classes which you've listed, i dropped school somewhere halfway through. I know how to write code for ML, this isn't totally new to me.(by this i mean i know how to use the python ml library)
i ask topically, what math is most important in the further development of ai?(ie complex analysis, combinatorics, graph theory, stochastic methods are all things important in ML, what other math is worth noting?)
the math you listed are simply things required for most comp-sci majors.
ah ok. I am also starting with ML, so take this with a grain of salt.
While the courses i listed are required for most CS majors, i think they are more important for ML than many other fields of CS (of course there are exceptions). I think a deeper understanding of LA, Calculus and Numeric compared to normal CS-majors is beneficial (but if you majored in math you probably had to take these in more depth than most CS-majors). Also i would double down on Probability (so a course on Probability-Theory, Markov-Chains etc.). Optimisation Theory is also important, but often already somewhat covered in the other courses. I don't think one can point to a specific course that will be really important for NN in the future, nobody really knows whats coming. Just learn as much math as possible, so that you are prepared to learn and understand all the future theory and techniques that are not yet discovered. Maybe we will all be solving stochastic differential equations in the future, because they accurately predict the performance of various NNs?
Edit: I forgot tensor-mathematics, i haven't taken a course yet, but it seems pretty straightforward.
awesome, thank you!
I took linear an analytic linear algebra class some years ago, but admittedly don't remember too much. I'm beginning to study more in depth matrix analysis, this and combinatorics are useful for understanding fundamentals of many algorithms.
I've learned a little of Markov Decision Process, but i've been told by people working in the field that MDP isn't used much for neural net anymore, though this doesn't mean it shouldn't be learned. I'll definitely look into tensors.
i was also told that Markov-chains are not used for NN anymore, but i learn them because often they get combined with NNs and used for other aspects of the problem (if i remember right AlphaGo used them beside NNs).
the blog article was useful, thank you! A problem with most of the recommendations is that they are targeted at software engineers wanting to learn some ML. They mostly don't need to understand back propagation fully and don't have that complex of a problem. So the requirements are the absolut basics (what are vectors/matrices and how to multiply them). Also their math-background is either non-existent or a long time ago. I am a student and constantly learning math, so i am in a different position. I am also interested in really understanding NN, including recent papers. They vary, some being not very math-heavy (the dropout paper) and some being the opposite (bayes neural network, only skimmed some papers). Many paper introduce a new idea borrowed from other branches of math, so you can't specialise too much. A broad understanding is important. Also we still lack a unifying theory, so we definitely don't know what math will become really important in the future.
In the current system, yes, properly implemented, UBI would end poverty in America while still allowing for massive inequality. It sets a floor, not a ceiling.
I'm not sure that inequality is necessarily the thing to be fixed.
I suspect that when most people talk about inequality, what they mean is, "a few people are doing really good and a lot of people are doing really bad". If you can lessen or remove the "doing really bad" part, whether through UBI or some other means (progressive taxation to fund other programs, whatever your ideology prefers), then statements of inequality start to mean just "a few people are doing really good" and I think there are far fewer people that are bothered by that.
I also think these conversations tend to get stuck on the definition of poverty. Some people say, "well, it can't be poverty because they all have smartphones", and other people say, "well, it's poverty because they lack any opportunity". I think that if these discussions are going to get anywhere we have to start by convincing people that the definition of poverty is allowed to change and that we shouldn't in the 21st century be defining poverty in 19th-century terms.
>I suspect that when most people talk about inequality, what they mean is, "a few people are doing really good and a lot of people are doing really bad". If you can lessen or remove the "doing really bad" part, whether through UBI or some other means (progressive taxation to fund other programs, whatever your ideology prefers), then statements of inequality start to mean just "a few people are doing really good" and I think there are far fewer people that are bothered by that.
I'd say the problem with inequality is something else entirely than "some people doing really good" vs "lots of people doing really bad".
It's that there are resources in the world, and some people have way more access to them.
This can hold even when all people are out of poverty: of the potential and resources of the world, still some people could take the lion's share to the exclusion of others, regardless of whether everybody has a house and food on their table.
And this inequality (with the true sense of the word, not mere poverty vs richness) is also related to opportunities and squandered potential (as in: yeah, we all have UBI, but this dumb mega-rich person could pay his way to Harvard, whereas some other with much larger potential -- e.g. smarts, inventiveness, etc-- had to settle for some lesser school, and didn't get as far in their career).
There's a scale of ideology here from, "everyone should be guaranteed to get exactly the same as everyone else regardless of luck and effort" to "life should come with no guarantees and everything is down to luck and effort".
It sounds like you're further to the left of that scale than I am, which is fine, except that it's harder to convince people to the right of me on that scale to move that far left.
Poverty is a problem, obviously, but inequality is a problem in itself.
* inequality subverts the democratic process
* inequality subverts the rule of law
* I'd argue it even threatens human dignity
* in economics, my utility function typically only takes my well-being/wealth/consumption/circumstances into consideration, not my neighbour's. That's a simplification, so that one can do economics. It's pragmatic and expedient - it is certainly neither descriptive nor normative. Although there is a tendency in libertarian/"economism" circles (google James Kwak "economism" if you don't know what it means) and even others (skeptic Michael Shermer, for example) to consider it the rational thing to do.
I don't see why. Yes, if I get 10$ more, and all my neighbours get $10000 more, I will be cranky and unsatisfied, utility function bla bla my ass.
Inequality also depresses human happiness.
TL;DR: excessive inequality jeopardizes democracy, the rule of law, human dignity, peace, and happiness.
Assuming you implemented UBI, and changed nothing else, what would prevent the market from adjusting overall price levels (upwards) until those living only on an UBI again end up below the (now increased) poverty line?
In other words, I believe that without penalizing the upper end, too, things will just "adapt" to define a new lower end, if only the only change is to give everybody a little bit of money. To some extent, Germany has been doing something close to UBI for quite a while now ("Grundsicherung", "Harz IV", etc.). It certainly helped prevent that a few people fall into some nasty global minimum with zero income, so its probably better than not having it. But I don't think that only clamping the lower income end alone had a society-transforming impact in Germany.
You can order the standard machine learning texts from most to least math-y, and least to most modern:
- Pattern Recognition and Machine Learning (Bishop 2007)
- Machine Learning: A Probabilistic Perspective (Murphy 2012)
- Deep Learning (Goodfellow, Bengio, Courville 2016)
If you want cutting-edge material, read the Deep Learning book (which is still quite technical, though some of its content may be outdated in a few years). If you want timeless mathematical foundations very clearly presented, read Bishop. Murphy is a good middle ground.
If you're self-teaching and have trouble focusing on a textbook for long periods, the Stanford CS229 lectures combined with Andrew Ng's course notes and assignments are probably the best resource. They are still quite rigorous and working through them will give you a solid foundation, after which you'll be more prepared to understand the deeper content in any of the texts above.
I didn't expect to see Hopcroft/Kannan in that list. I used it in a course taught by Kannan and back then it was called "Computer Science Theory for the Information Age". Apparently the book is now called "Foundations of Data Science". I always thought the old naming was terrible but compared to the new title it actually describes the content a lot better since the book is mainly about CS theory and mathematical foundations for them. It was one of my favorite courses but I would not classify it as ML book.
I hear people recommend textbooks a lot, and I honestly don't know why. Very few researchers I know learned machine learning through reading a textbook on their own. Furthermore, the first two textbooks are closer to reference books rather than actual pedagogical tutorials (I haven't read any single chapter in its entirety). The Deep Learning book assumes machine learning knowledge.
On a similar note, Udacity sprung up from the AI Class that ran at the same time (taught by Peter Norvig and Sebastian Thrun); Udacity has since added the class to their lineup (though at the time, they had trouble doing this - and so spawned the CS373 course):
I took the CS373 course later in 2012 (I had started the AI Class, but had to drop out due to personal issues at the time).
Today I am currently taking Udacity's "Self-Driving Car Engineer" nanodegree program.
But it all started with the ML Class. Prior to that, I had played around with things on my own, but nothing really made a whole lot of sense for me, because I lacked some of the basic insights, which the ML Class course gave to me.
Primarily - and these are key (and if you don't have an idea about them, then you should study them first):
1. Machine learning uses a lot of tools based on and around probabilities and statistics.
2. Machine learning uses a good amount of linear algebra
3. Neural networks use a lot of matrix math (which is why they can be fast and scale - especially with GPUs and other multi-core systems)
4. If you want to go beyond the "black box" aspect of machine learning - brush up on your calculus (mainly derivatives).
That last one is what I am currently struggling with and working through; while the course I am taking currently isn't stressing this part, I want to know more about what is going on "under the hood" so to speak. Right now, we are neck deep into learning TensorFlow (with Python); TensorFlow actually makes things pretty simple to create neural networks, but having the understanding of how forward and back-prop works (because in the ML Class we had to implement this using Octave - we didn't use a library) has been extremely helpful.
Did I find the ML Class difficult? Yeah - I did. I hadn't touched linear algebra in 20+ years when I took the course, and I certainly hadn't any skills in probabilities (so, Kahn Academy and the like to the rescue). Even now, while things are a bit easier, I am still finding certain tasks and such challenging in this nanodegree course. But then, if you aren't challenged, you aren't learning.
I haven't read it in detail, but my impression is that it is mathy, like Bishop, but focuses more on 'classical' frequentist analysis, whereas Bishop takes a more open-ended Bayesian perspective and covers important machinery like graphical models and inference algorithms that I don't think are in ESL.
As a researcher I tend to prefer the Bayesian perspective in Bishop, because it gives you a unifying framework for thinking about building your own models and learning algorithms. But lots of people seem to respect ESL and speak very highly about it. It's probably valuable if you are implementing one of the methods it covers and want to understand that specific method in great depth.
I appreciate the comparison. My field is computational biology, and the latest edition of ESL has some specific, relevant examples, which is what drew me to it in the first place. I'm currently working on getting my statistics background up to par, and then I'll be choosing an ML textbook. However, your note about ESL focusing on frequentist models gives me pause about my original choice, as it would seem to me biological applications naturally lend themselves to Bayesian methods. I think in the end I'll have to crack open several books to decide. Thanks for the input.
You're welcome! FWIW I mostly agree with argonaut's point elsewhere in this thread - very few people successfully self-teach ML from a textbook alone. So whichever book(s) you choose, it might also be worth working through some course materials. I've already suggested Stanford's CS229 for solid foundations, but depending on your interests in bioinformatics, Daphne Koller's Coursera course on probabilistic graphical models (https://www.coursera.org/learn/probabilistic-graphical-model...) might be especially relevant. Koller literally wrote the book on PGMs, has done a lot of work in comp bio, and her MOOC is apparently the real deal: very intense but well-reviewed by the people that make it through.
Not sure why this is downvoted. ISLR is widely considered the "easier to read" / more pedagogical version of ESL. It's still mathematical but is closer to what the poster wants - a textbook that teaches them.
Interesting, thanks - I wasn't aware of this text.
But according to your link, ESL isn't a prototype of ISL, but a more 'advanced treatment'. The R applications in ISL seem like they might be very useful, though.
Also consider Foundations of Machine Learning (2012) by Meryar Mohri. It is a very good book with a statistical learning approach to ML. If you're interested by statistical guarantees to ML algorithms, eg. why the SVM algorithm works and its sample complexity (the expected out of sample error in function of the sample size), it's the book you need.
The main reason we see all these relatively information-less blog posts / tutorials is because most people on HN are afraid of the math. If you're not afraid of the math (or have a relatively strong math background), then honestly you can jump into the CS 229 Stanford lecture notes and homework assignments. If that's a little too heavy for you, you can also start with the MOOC version of that course (which is a very watered down version of CS 229).
