Pedestrian detection bias

This page contains a dataset for evaluating bias in pedestrian detection algorithms. Specifically, it contains age and gender labels for bounding boxes in the INRIA Person dataset (test set only).

The dataset is described in the following publication:

  • M. Brandao, Age and gender bias in pedestrian detection algorithms, in Workshop on fairness accountability transparency and ethics in computer vision, cvpr, 2019.
    [Abstract] [BibTeX] [PDF]

    In this paper we evaluate the age and gender bias in state-of-the-art pedestrian detection algorithms. These algorithms are used by mobile robots such as autonomous vehicles for locomotion planning and control. Therefore, performance disparities could lead to disparate impact in the form of biased crash outcomes. Our analysis is based on the INRIA Person Dataset extended with child, adult, male and female labels. We show that all of the 24 top-performing methods of the Caltech Pedestrian Detection Benchmark have higher miss rates on children. The difference is significant and we analyse how it varies with the classifier, features and training data used by the methods. Algorithms were also gender-biased on average but the performance differences were not significant. We discuss the source of the bias, the ethical implications, possible technical solutions and barriers to "solving" the issue.

    @INPROCEEDINGS{Brandao2019fatecv,
    author = {Martim Brandao},
    title = {Age and gender bias in pedestrian detection algorithms},
    booktitle = {Workshop on Fairness Accountability Transparency and Ethics in Computer Vision, CVPR},
    year = {2019},
    month = {Jun},
    abstract = {In this paper we evaluate the age and gender bias in state-of-the-art pedestrian detection algorithms. These algorithms are used by mobile robots such as autonomous vehicles for locomotion planning and control. Therefore, performance disparities could lead to disparate impact in the form of biased crash outcomes. Our analysis is based on the INRIA Person Dataset extended with child, adult, male and female labels. We show that all of the 24 top-performing methods of the Caltech Pedestrian Detection Benchmark have higher miss rates on children. The difference is significant and we analyse how it varies with the classifier, features and training data used by the methods. Algorithms were also gender-biased on average but the performance differences were not significant. We discuss the source of the bias, the ethical implications, possible technical solutions and barriers to "solving" the issue.},
    topic = {Robot ethics},
    url = {http://www.martimbrandao.com/papers/Brandao2019-fatecv.pdf}
    }

You can download the data here: Pedestrian detection bias. Please cite the publication above if you use the dataset.