The design is evaluated based on accuracy and throughput.


Accuracy is measured using the F1-score, where:

F1-score = (2 * precision * recall) / (precision + recall)

precision = global_true_positives / (global_true_positives + global_false_positives)

recall = global_true_positives / (global_true_positives + global_false_negatives)

True positives are when a reported object matches the type and location (IoU > 0.5) of an object from the golden data.

The minimum accuracy should be TBD, otherwise a penalty is applied.


Throughput is measured in FPS. The time to read images from the SD card is not included.

The design should achieve at least 5 FPS (subject to change), otherwise a penatly is applied.


The score for a team is calculated as follows (Subject to change):

team score = F1-score^2 * fps

Scoring Script

A score.py script is provided that will provide f1-score and fps.