This is a demo of MLTest on the dataset
marmal88/skin_cancer.
Each image is labeled with one of three different kinds of cancers.
The model has been trained on five models: two variants of Swin Transformers, ViT, ResNet, and BEiT. The test results for each model can be inspected in the dashboard below.
Performance tests: in order to measure how well the model performs, we compute common performance metrics like accuracy, precision, recall, F1 score, and more.
Failure clusters:
these clusters give meaningful insights when the model is failing and can be inspected in the "Failure Clusters" tab.
These failure clusters are automatically detected
for different combinations of metadata.
For example, the BEiT transformer has a significantly lower accuracy on images taken of cancers of the back with class label 0.
Robustness: these tests help ML developers evaluate how well their model performs under different conditions. These conditions could include different levels of brightness, compression, and many other types of interference.
The following robustness tests were enabled for this test case:
The full list of transforms supported by MLTest can be found in the documentation.
Fairness tests: these tests measure how fair your model is. That means, whether its performance is dependent on a protected attribute of a person. In this dataset, the age and gender of a subject may be considered protected attributes.
We used two types of fairness tests on the age and gender of a person.
The Equalized Odds test checks that true positive and false positive rates are equal amongst protected attributes.
The Predictive Equality test checks that the false positive rates are equal amongst protected attributes.
More fairness tests supported by MLTest can be found in the documentation.