Core Concepts
Representing multivariate functions as trees to uncover interaction effects.
Abstract
The article introduces function trees as a method to represent multivariate functions, emphasizing the importance of understanding interaction effects in machine learning models. It discusses the construction of function trees, their application in various datasets, and compares them with other modeling techniques like MARS and XGBoost. The focus is on interpreting complex models for better insights.
Stats
"The output of a machine learning algorithm can usually be represented by one or more multivariate functions of its input variables."
"A method is presented for representing a general multivariate function as a tree of simpler functions."
"Interaction effects involving up to four variables are graphically visualized."
"There are 10000 observations with outcome variables generated as y = F(x) + ε with x ∼ N 8(0, 0.5)."
"The noise is generated as ε ∼ N(0, var(F)/4) producing a 2/1 signal/noise ratio."
Quotes
"The most accurate function approximation methods tend not to provide comprehensible results."
"Function trees expose the global internal structure of the function by uncovering and describing the combined joint influences of subsets of its input variables."
"Partial dependence functions choose a compromise in which the variables in z are taken to be independent of those in ˜z."