[MLE] Introduction

Some content had wrote before but I would write it again for the completeness

What’s Machine Learning?

Two definitions of machine learning are offered:

  • Arthur Samuel described it as: “the field of study that gives computers the ability to learn without being explicitly programmed.” This is an older, informal definition.
  • Tom Mitchell provides a more modern definition: “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.”

Machine learning aim

Learning functions: Given a set of training data consisting of pairs (x, f(x)), where x are called features and f (x) = y the labels, Machine learning (ML) is the process of learning a function, or hypothesis, h(x) -> yˆ that best approximates the unknown function f(x) .
This ability to make correct predictions on unseen data is called generalisation.

Representation view on ML

Learning = Representation + Evaluation + Optimisation

Pattern Recognition VS Machine Learning

Definition

  • Machine learning is learning from experience. It’s also called supervised learning, where E is the supervision
  • Pattern Recognition is finding patterns without experience. It’s also called unsupervised learning
    For further explanation of supervised and unsupervised learning, see My blog here

Supervised Learning terms

Classification is a ML task where T has a discrete set of outcomes: h(x) -> y ∈ N

Regression is a ML task where T has a real-valued outcome on some continuous sub-space: h(x) -> y ∈ R

Features and labels

  • Data points or instances are the data used to learn a hypothesis h or find a pattern g
  • In Machine Learning, data points consist of feature/label tuples {x,y}

labels are the values that h aims to predict
Features and Attributes are measurable values of variables for which some form of pattern exists, that can be used to infer the associated label y

Regression problem is a continuous problem!

enter image description here

Representation of our training set

enter image description here

评论

发表评论

此博客中的热门博文

[MLE]Decision Trees

图片
Overview Decision Trees Linear vs non-linear classifiers Entropy C4.5 Random Forests Some fundamental concepts The process of selecting a specific model, given a new input x, can be described by a sequential decision making process corresponding to the traversal of a binary tree (one that splits into two branches at each node). Here we focus on a particular tree-based framework called classification and regression trees Basic Decision Trees Decision trees apply a sequence of linear decisions, that often depend on only a single variable at a time. Such trees partition the input space into cuboid regions, gradually refining the level of detail of a decision until a leaf node has been reached, which provides the final predicted label As the above figure indicates, we should follow the rules for tree classification: Start at root node Evaluate relevant attribute of that node Follow branch according to attribute evaluation to descendent node Descendent node can then be co
阅读全文

[AIM] MetaHeuristics

图片
Day2 Outline Metaheuristics -definition Single point based search method(simple stochastic local search) Iterated local search Scheduling Tabu Search Metaheuristic what is a metaheuristic? A metaheuristic is a high-level problem independent algorithmic framework that provides a set of guidelines or strategies to develop heuristic optimization algorithms. what is the difference between heuristic and metaheuristic? Heuristic : Heuristics are problem-dependent techniques. As such, they usually are adapted to the problem at hand and they try to take full advantage of the particularities of this problem. However, because they are often too greedy, they usually get trapped in a local optimum and thus fail, in general, to obtain the global optimum solution. Metaheuristic : Meta-heuristics, on the other hand, are problem-independent techniques . As such, they do not take advantage of any specificity of the problem and, therefore, can be used as black boxes . In general, t
阅读全文

[MLE] Linear Classification

图片
Linear Classification Discriminant functions Least squares for classification Fisher’s linear discriminant K-Nearest Neighbour(KNN) Classification What’s decision boundaries/linearly separable? The goal in classification is to take an input vector x and to assign it to one of K discrete classes \(C_k\) where k = 1,…,K. In the most common scenario, the classes are taken to be disjoint, so that each input is assigned to one and only one class. The input space is thereby divided into decision regions whose boundaries are called decision boundaries or decision surfaces. In this blog, we consider linear models for classification , by which we mean that the decision surfaces are linear functions of the input vector x and hence are defined by (D-1) -dimensional hyperplanes within the D-dimensional input space. For example: in 3D input space, the decision boundary will be a 2D surface What is Linear Separability Linearly separable data: Datasets whose classes can be separated by
阅读全文