Automatic testing for ML pipelines is hard. Part of the executed code is a model that was dynamically trained on a fresh batch of data, and silent failures are common. Therefore, it’s problematic to use known methodologies such as automating tests for predefined edge cases and tracking code coverage.
We’ll demonstrate common pitfalls with ML models, and cover best practices for automatically validating them: What should be tested in these pipelines? How can we verify that they’ll behave as we expect once in production?
We’ll discuss how to automate tests for these scenarios, introduce the deepchecks open source package for testing ML models and data, and demonstrate how it can aid the process.
How to Efficiently Test ML Models & Data
Born and raised in Nahariya, Israel, Chorev is a former member of the Israel Defense Force's "Talpiot" program for technological leadership, as well as a member of the famous intelligence agency, Unit 8200. Now monitoring AI, rather than humans, Deepchecks' software continuously validates machine learning systems' performance and stability. The company has raised $4.3 million so far from investors like Grove Ventures and Hetz Ventures.