Support

The need for Data Versioning

Data versioning can be applied to the Data Feature set Map, as well as to the new data version that is created as a result of the new settings of the encoder.

CoreAI will automate that the second dataset will be versioned by the first dataset version and the committed code of the autoencoder model.

coreControl is data centric. Therefore, we put emphasis on how to interact with the data. The main issue with data versioning systems is that they don't give you a clear way to visualize what is going on. After you version your data, it is hard to see what is the actual difference between those versions. With current solutions you have to download every different version and manually take a look. This is very tedious and time consuming.

With coreControl, for every version of the data, you will be able to see an overall description of the data like the number of samples & features, the storage it uses and the newest samples that were added. You can easily take a look at statistics properties like mean, median, quantiles and more.  Also, for every feature we compute the outliers, histogram and distribution.

Experiment Management / KPIs

One of the key factors of an MLOps infrastructure is to track the experiments. After dozens or even hundreds of experiments, it is difficult to remember which hyperparameters performed best and which one had the leading KPI. coreControl can easily be integrated into the training code of your model to keep track of your configuration, hyperparameters, and metrics. Also, it has support to log different kinds of objects like images, tables, graphs, etc.

Another issue appears after you have logged information for dozens of experiments and you want to compare them. coreControl’s simple interface gives you the possibility to pin a couple of key metrics that will show up directly into the table where you search for experiments. In this manner, you will be able to easily surf between experiments and pick the desired one.

One last thing to mention is that you can implement any custom KPI/metric, both model and business-wise. Hence, you will always feel in control.

Model Monitoring

While your model is in production, it is the subject to degradation. Your model is prepared to handle situations similar to those it was trained on. Hence it won’t respond well to data that is different from the one used for training.

Moreover, as the model degrades, it requires careful monitoring to ensure that the prediction performance is not affected by the passage of time or other unexpected events. Therefore, robust production models require advanced monitoring that is usually divided into data monitoring and concept drift. Additionally, success criterias are monitored as the model is in service, which will help validate better model versions success.

Data Monitoring

Many of the ML/AI models are based on time series data and require continuous training of models to keep it up to date. As your model is already in production, an automated model update or a manual one, require monitoring of the data to make sure it is keeping the same assumption you made in your research phase.

Graphical representation of the various data, as well as valuable out of distribution events. Notification of drift, anomaly, and untrusted situations.

With a robust data monitoring system, you are able to visualize all the data that is different from your training scenario. coreControl offers various methods that detect this drift like KL divergence, KS statistical test, tree methods, etc. Also, with an alarm system in place, you will be able to catch those scenarios in time and change them properly.

In the end, with a proper monitoring system, you will always be prepared to adapt to new changes, quickly react to unexpected events, mitigate risks, and constantly bring value to your clients.