Decision Trees Finalized

Previously, we established learning as the ability to acquire knowledge. Further, I introduced the concept of decision trees as a basic model for machine learning. We've also differentiated between types of problems suited to machine learning as well as types of ML algorithms. With that in mind, I want to finalize our decision tree discussion. Remember, decision trees are a great introduction to programming machine learning algorithms. Let's dig in!

Brief Recap

We've been examining a binary classification algorithm that predicts whether a student will like a programming course. The algorithm is supervised. Supervised algorithms use a training dataset.

The specific use came we have been developing is a classifier for programming courses. Thus, we can ask will students like a programming class based on prior experience with programming and prior experience with course scheduling (i.e, time of day)?

Our Course classifier

We already outlined our training dataset. Accordingly, we're left with the secret sauce. Here, the secret is in how we traverse the structure of the tree. I'm referring to the idea of splitting.

Splitting is the action taken when we reach an interior node or feature. When we split, we're sorting data into yes and no subsets represented by the decision tree leafs or end nodes. More precisely, we're splitting into subsets that represent the binary classification categories represented in our question (the implicit yes or no associated with will a given student like a specific programming class?).

Let's look at a general pseudocode example.

if prior_experience = 'yes' AND course = 'programming'
    THEN label = 'yes'

if time = 'night' THEN label = 'no'

if prior_experience = 'no' AND course = 'programming' 
    THEN label = 'no'

Such a function clearly isn't complete. The function only represents training. However, we've demonstrated the simple logic involved.

Now, we, as the programmers, define the rules associated with how the algorithm makes decisions (that is, splits and classifies. Further, I don't think we've diminished the technological power imparted by using decision trees despite demystifying the supposed intelligence of such a program.

Moving Forward

That, my friends, is a basic supervised binary classification decision tree. Certainly, there is more to the programming implementation than what we've looked at here. However, I think we've seen enough to dispel some of the mystery surrounding machine learning. There is a great deal more to reveal too. Join me next time as we expose some details of how our algorithms calculate things like cost of a decision and information gain.