While working on modeling the behavior of a component of the HVAC system using machine learning algorithms, I observed that when the data was changed to observe how the system would react, there were sharp movements in the model that didn't quite map out to reality.
 |
| Source: https://mindsports.nl/index.php/moving-forward-looking-back |
The issue bugged me. I knew there would be a solution.
One day, while I was having my morning coffee and reading a paper on path dependence in economics, I saw the solution in front of my eyes: to perfectly capture the behavior of the system, not only do I have to feed the algorithm the present data, but the previous (t-1) data as well!
Rushing to my laptop, I fixed the code to account for previous data as well as present data, and voila! It worked! I could easily replicate the behavior of the system with this very simple step.
Often times, data scientists work with data they are not familiar with, which leads to over-complications in the code/algorithm to reach the desired results, with little room for flexibility in the future.
But as engineers equipped with data science skills, simple solutions can unlock complex problems.
Cheers,
Edmond